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ABSTRACT 



This report presents the findings from a 30-month evaluation 
of the Job Training Partnership Act (JTPA) Title IV Section 401 Indian and 
Native American (INA) Program. This specially targeted JTPA program provides 
employment and training services to American Indians, Alaska Natives, and 
Native Hawaiians. Most grantees are tribal governments or community- based 
organizations. The evaluation consisted of a survey completed by 113 of the 
approximately 175 INA grantees and site visits to 23 randomly selected 
grantees during which services were observed and staff and program 
participants were interviewed. Following an executive summary and an 
introductory chapter, the study design is detailed in chapter 2. Chapter 3 
presents quantitative data from the grantee survey. Chapter 4 takes a 
broad-brush look at grantee organizations and service strategies, including 
goals and objectives, funding, staffing, and service emphases. Chapter 5 
looks in depth at specific elements of service delivery: outreach and 
recruitment, the processes by which participants' needs are assessed and 
their service strategies developed; and the use of supportive services and 
stipends to support participants through training. The next four chapters 
focus on basic and occupational skills classroom training, on-the-job 
training, work experience and community service employment, and training 
assistance and placement services. Chapters 10-11 present factors influencing 
program design decisions, an assessment of program effectiveness, and some 
policy recommendations. Three appendices contain the survey, client 
characteristics and outcomes, and a list of abbreviations. (TD) 
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EXECUTIVE SUMMARY 



This report presents the findings from a three-year study of the JTPA Title IV 
Section 401 Indian and Native American (IN A) Program. The program is designed to 
deliver employment and training services to Indians and Native Americans throughout 
the nation, through a network of grantees that includes tribal governments and other 
Indian organizations. The study was designed to examine the goals and objectives 
programs have established for themselves, the service design decisions that resulted, 
constraints and challenges in implementation, and the overall quality and effectiveness 
of the services in meeting the needs of participants and communities. 

Background of the Evaluation 

The very unique historical, socio-economic, and cultural context of Native 
American communities has provided ample justification for a specially-targeted 
program under JTPA, and under JTPA’s successor, the recently enacted Workforce 
Investment Act (WIA) of 1998. These unique circumstances include extraordinarily 
high rates of joblessness and discrimination directed against Native Americans, even in 
the face of America’s recent robust economy; the importance of tribal sovereignty that 
should give Native American grantees a greater measure of discretion in running their 
programs; unique cultural values and a sense of shared identity that provide meaning to 
Indians’ lives; and tribal areas that are often physically remote, with incomplete 
infrastructures, and very limited labor markets. 

In this context, Section 401 of JTPA calls for a specially-targeted program for 
serving the Indian and Native American population. With annual funding currently at 
about $50 million, the program serves approximately 19,000 participants each year 
through a network of about 175 grantees that includes tribal governments, tribal 
consortia, non-tribal Native American organizations, and, sometimes, agents of state 
governments. These grantees vary greatly in size and in the contexts in which they 
operate, and include programs with very limited JTPA funding serving narrowly 
circumscribed service areas, as well as organizations receiving hundreds of thousands 
or millions of dollars in annual funding and serving vast sweeping reservations, large 
metropolitan areas, or even entire states. Taken together, these grantees are 
responsible for providing services to eligible Indians and Native Americans throughout 
the United States. 
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To guide our study of the INA program and provide a framework for the 
evaluation, we developed a heuristic model of service quality. This model depicts how 
clients flow through the system and shows the quality indicators for each step in the 
process. The model suggests that programs need to provide, either directly or through 
their service providers: effective outreach and recruitment mechanisms, so that all 
eligible persons who are interested in services can have access (within the constraints of 
available funding); an initial assessment that is appropriate to the clients’ needs and 
purposes and identifies barriers to successful participation; effective matching to 
services and case management to ensure that clients are assigned to appropriate services 
and receive the support they need to see that their service plan is successfully carried 
out; training that is of the highest quality, including well-specified and individualized 
learning objectives and contextualized instruction using active learning methods; and 
training assistance that, to the extent that opportunities in the local area allow, promote 
high quality job placements and build on skills that were attained. Outcomes need to 
include, for the participant, the opportunity to learn new skills, improved self-esteem, 
exposure to the labor market, increased employability, and the advancement of both 
short-term and longer-term career and other objectives. Community-level outcomes are 
also of paramount concern to INA grantees, including efforts to improve local services 
and build capacity for Native American communities and promote a strong sense of 
culture and community identity. 

Study Design 

The evaluation consisted of two primary components. The first of these was a 
mail Administrator Survey sent to all grantees. The survey instrument was developed 
in close consultation with representatives of the grantee community, the U.S. 
Department of Labor (DOL), and the Office of Management and Budget, and included 
sections on program goals and objectives; perceived obstacles to obtaining successful 
outcomes for participants; and an assessment of the nature of relations with the 
Department of Labor, including grantees’ appraisal of the clarity of DOL’s guidance, 
the adequacy of the consultation process and the training and technical assistance that is 
provided, satisfaction with performance standards and reporting requirements, and so 
on. Survey administration began in early 1998 and extended through the spring of that 
year. In addition to asking grantees to provide responses in close-ended categories, we 
also provided them with the opportunity to write in their thoughts on a wide range of 
topics. Altogether, we received completed returns from 113 grantees, for a response 
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rate of about 70%. We were gratified to see that almost all respondents took the time 
to write extensive comments throughout all sections of the survey. 

The second and much more elaborate of the evaluation’s two components 
consisted of site visits to 23 randomly selected grantees across the country. The 
purpose of these case studies was to complement the Administrator Survey by gaining 
in-depth knowledge about a subset of grantees, understanding the reasons for the 
service design decisions that were made, identifying constraints and challenges in 
service delivery, and assessing the quality and responsiveness of the services that were 
provided. The site visits lasted 2-3 days each and consisted of interviews with program 
administrators and staff, program participants and recent terminees, and allied 
community organizations. While on site, field researchers also observed several 
instances of service delivery (e.g., a classroom or worksite activity) and spoke with 
those delivering the services (e.g., classroom instructors or worksite supervisors). In 
selecting the 23 grantees to visit, we developed a stratified random sampling plan that 
ensured representation of tribal and non-tribal programs, those operating in various 
regions of the country, large and small programs, and those operating in varying socio- 
economic contexts. 

Results from the Administrator Survey 

The results from the Administrator Survey are presented in three sections, 
discussing relationships with the Department of Labor, program goals, and factors 
affecting clients’ success. 

Relationships with the Department of Labor 

Grantees interact most directly with DOL through DOL’s Division of Indian and 
Native American Programs (DINAP), which provides overall direction and guidance. 
As part of the Administrator Survey, grantees by overwhelming proportions (over 90%) 
stated that DINAP clearly communicated program policies while providing enough 
flexibility for grantees to accomplish their chief objectives. They also gave extremely 
high marks to the recent partnership initiative instituted by DINAP, finding that it was 
now a lot easier to “have a say in program policies” and had established an atmosphere 
of trust and mutual respect. Although the consultation process is thus viewed very 
favorably overall, a number of respondents suggested that a wider mix of grantees 
might be invited to participate in various Work Groups and advisory committees. 



Among its other duties, DINAP also co-sponsors national and regional training 
conferences and, through other means, is charged with providing grantees with the 
technical assistance they need to be effective. Grantees overwhelmingly believe that 
their training needs are being met through these efforts. The opportunities that grantees 
have to meet one-on-one with their Federal Representatives are much appreciated, and 
DINAP staff are widely praised for their helpfulness and timely assistance. The 
national and regional conferences are similarly viewed very favorably, and the DINAP 
Bulletins are felt to provide important information. Among vehicles for technical 
assistance, peer-to-peer exchanges are strongly endorsed. 

Despite this very favorable response overall, grantees did make numerous 
suggestions for how their technical assistance needs could be better met. Suggestions 
included providing more comprehensive training for new Directors, tailoring 
workshops at conferences to subsets of grantees (e.g., based on the size of the grant, 
for reservation vs. urban programs, for more vs. less experienced staff, etc.), providing 
updated manuals, and improving access to electronic conferencing. 

Reporting and performance standards are two vehicles used to ensure 
accountability. Grantees almost unanimously agree that recent changes to the reporting 
forms make them easier to use and they believe that the required forms generally cover 
all information of relevance that needs to be reported. Similarly, DINAP’s recent 
moves to encourage electronic reporting are generally well received, although many 
grantees are reserving judgement until they see how well the transmission procedures 
will work in practice. However, an appreciable minority still feel that the reporting 
forms are too complicated. 

Sentiments are more divided on the performance-standards system. Between 70% 
to 85% of respondents understand the current system of performance accountability and 
are satisfied with it overall, and believe that it helps them improve their program 
performance and that the process of adjusting standards is fair. Moreover, two-thirds 
believe that the current measures of outcomes give enough flexibility for grantees to run 
their programs as they see fit. At the same time, majorities also feel that the 
performance-standards system is too complicated, focuses too much on job placements, 
and does not adequately “reflect the goals of our program.” 

Thus, although the current system may be something that most grantees can live 
with and that some even prefer, sentiments are clearly mixed. Accordingly, grantees 
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had numerous suggestions for improving performance measurement, such as that a 
wider range of outcomes should be covered, that grantees should be able to “claim 
credit” for services they provided (and not just outcomes), that measures should be 
developed relating to learning gains for participants who have not yet terminated, and 
that grantees should be permitted to develop individually-designed performance 
measures tailored to their own programs. Some of these suggestions are in line with 
recommendations recently issued by a Work Group convened by DOL to assist in 
revamping the current performance measurement system. 

Program Goals 

Section 401 programs operate in extraordinarily difficult circumstances, and 
grantees find themselves trying to meet the needs of hard-to-serve program participants 
in often bleak socioeconomic contexts. Given these circumstances, they must balance 
diverse program objectives, from providing training that boosts long-term 
employability, to providing a source of income in the short-run, and from meeting the 
needs of individual participants, to enhancing the well being of the community as a 
whole. 

These diverse objectives were reflected in responses to the Administrator Survey. 
Respondents nearly unanimously define their objectives as including quite traditional 
employment and training functions, such as providing participants with training and 
helping them find jobs and achieve self-sufficiency. But additionally, majorities also 
rate helping participants with temporary income and supportive needs as important, as 
well as providing a sense of community. The importance of community as a program 
objective is especially important for non-tribal programs, presumably because other 
community agencies serve this function in areas served by tribally-run programs. 

Factors Affecting Clients’ Success 

In recognition of the difficult circumstances within which Section 401 programs 
operate, we asked grantees to identify factors that made it difficult for clients to achieve 
successful outcomes. External constraints cited as being most important were the 
grantees’ lack of funding and a lack of adequate job opportunities in the area. Client 
characteristics that were noted as among the most formidable constraints included 
participants’ lack of work experience, lack of motivation, problems with substance 
abuse, family responsibilities, lack of awareness of what it takes to get and keep a job, 
and lack of necessary life skills. Many clients also are strongly tied to their 
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communities, which makes them reluctant to pursue job opportunities that may require 
relocation. 

Grantee Organization and Service Strategies 

The in-depth case studies also provided the opportunity to look at programs’ goals 
and objectives and service strategies, and to understand how they were devised given 
the constraints of the local context. A content analysis of grantees’ goal statements 
showed a strong service orientation, in keeping with Native American cultural values 
that emphasize helping and serving others. Other key themes revealed a strong client- 
centered orientation that often led to a focus on attending to participants’ short-term— to 
some degree instead of their longer-term— needs. In general, however, goal statements 
are vaguely worded and lack specificity. 

An examination of service designs shows again an emphasis on meeting 
participants’ shorter-term needs. Thus, Training Assistance, which is usually designed 
for short-term goal attainment, is one of the primary services of choice. Work 
Experience is another service activity that is very heavily used, and this too often 
reflects an effort to attend to participants’ need for immediate income, as well as to 
provide services to the community that would otherwise be unaffordable. At the same 
time, service activities with longer-term training objectives were also a key part of the 
service mix in many programs, including occupational-skills classroom training, on-the- 
job training, and basic skills classroom training. 

Key factors influencing the service designs that individual grantees chose included 
the lack of job opportunities in many areas, which often led to a reliance on Work 
Experience and Community Service Employment; weak relationships in some cases 
with the private sector, due to employers’ overt racism or their poor appreciation of 
Native American cultures, which limited the use of OJT and restricted the range of 
suitable job placements; the vast geographic scope of many service areas, coupled with 
limited transportation networks, which minimized access to training providers and job 
opportunities; and limited and inadequate funding, which fostered a reliance on less 
expensive interventions. 

Limited funding, while thus circumscribing service options to a very real degree, 
nonetheless also added as a strong impetus for grantees’ efforts to link and coordinate 
with a variety of alternative agencies and funding streams. Indeed, many grantees have 
for many years made strong efforts to develop a “one-stop” approach to services, and 
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some have been quite successful in doing so. The specific form these efforts take 
varies, however. Thus, tribal programs are quite successful at accessing tribal funds or 
linking with other tribal agencies or programs specially targeted for Native Americans, 
but often shy away from forging strong linkages with state social service agencies for 
fear of compromising their tribal sovereignty. By contrast, non-tribal programs are 
much more likely to link with state agencies and do so quite successfully. Regardless, 
INA programs in our study have overall done a remarkable job of identifying an array 
of resources in their communities and accessing them to better meet the needs of their 
clients. 

With respect to staffing, research team members were profoundly impressed with 
the strong dedication and commitment found among most practitioners. In general, 
grantee staff were also seasoned veterans, well aware of their communities’ needs and 
available resources. However, some programs have experienced rapid staff turnover. 
Although new hires can often invigorate a program by bringing an infusion of new 
ideas, a lack of experience and institutional knowledge can sometimes be a hindrance to 
effective services. 

In any case, grantee staff uniformly understand the fundamental need for “Indian 
people to help other Indian people” and are strongly motivated along these lines. By 
the same token, the participants with whom we spoke expressed a strong preference for 
receiving services from Indian organizations, and indeed many would have avoided 
seeking assistance otherwise, however dire their needs. This fact suggests the crucial 
role that Section 401 programs play, especially in urban areas, of fostering community 
and serving as a critical lifeline for those in alienating surroundings. 

Service Delivery 

In examining the ways in which services were delivered in the 23 case-study 
grantees, we focused on services in support of training, basic skills and occupational 
skills classroom training, work experience and community service employment, and 
training assistance and placement services. 

Services in Support of Training 

Under this topic we considered the ways in which grantees targeted participants 
for services and carried out outreach and recruitment, conducted assessment and service 
planning, performed case management functions, and provided participants with 
stipends and supportive services. 



With respect to targeting and recruitment, most grantees avoid establishing 
specific targeting goals because they felt that all eligible INA applicants could be 
considered hard to serve and in need of assistance. For these reasons, grantees were 
reluctant to give priority to some individuals over others. In cases where targeting was 
an explicit goal, grantees focused on segments of the population deemed “most in need” 
by developing specific recruitment procedures, such as implementing a “no-repeaters” 
policy. While case study grantees reported little difficulty in meeting their enrollment 
goals, most relied heavily on indirect recruitment methods, such as word-of-mouth 
referrals or referrals from other agencies, although more proactive methods (e.g., 
advertisements, announcements at community events, job fairs, etc.) were also 
sometimes used. In some cases, an over-reliance on word-of-mouth referrals led to 
inadvertent targeting, whereby individuals with well-developed networks in the 
community could readily access services, while others who were not as well connected 
found out about the program only after an unwelcome delay. 

Assessment practices varied widely because of differences in the backgrounds and 
needs of participants. Thus, participants seeking training to improve their longer-term 
employability were often assessed carefully, including an examination of their basic 
skills proficiency, occupational aptitudes and interests, and supportive service needs. 
Meanwhile, those desiring an immediate job referral were assessed much more quickly 
and simply. To this degree, most grantees were careful about tailoring assessment 
activities to individual circumstances. However, even where extensive assessment was 
conducted, grantees were often constrained in developing fully effective service 
strategies, because clients often had strong opinions about what services they wanted 
and needed in the short-run, often to the detriment of their longer-term needs. 

Typically, therefore, service planning was driven by clients’ interests in obtaining 
immediate employment (subsidized or unsubsidized) and their need for immediate 
income. Additional constraints included grantees’ limited budgets and challenges posed 
by the service areas, such as the lack of transportation and the limited availability of 
service providers. 

Case management services in general were conducted as one-on-one counseling, 
though sometimes a team approach was used. In any case, case managers were 
genuinely caring and dedicated. We found many instances in which case management 
activities were carefully planned and well executed, and often grantee staff went to 
extraordinary lengths to ensure that their clients’ needs were met. Similarly, some 
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grantees devised strategies that entailed maintaining frequent communication with 
instructors, worksite supervisors, and other service provider staff to identify clients’ 
needs early and intervene appropriately. However, large case loads caused by limited 
resources, as well as geographically vast service areas, sometimes made case 
management difficult. Given these constraints, case managers often targeted their 
attention on participants facing difficulty or those just starting training, sometimes at the 
expense of those who appeared to be doing well. 

Basic Skills and Occupational Skills Classroom Training 

Nearly all programs offered classroom training of some type, but generally the 
emphasis was on occupational skills classroom training (OSCRT) rather than basic 
skills classroom training (BSCRT). Indeed, BSCRT was the service activity least 
common in the case study sample and was a service received by relatively few program 
participants, despite the fact that high school dropouts and those with low basic skills 
constitute a sizable part of the IN A program’s client pool. Part of the reason for the 
relative inattention given to BSCRT is that grantees often refer those interested in such 
training to adult basic education or GED programs run by other tribal departments or 
social service agencies, without enrolling them in JTPA. 

The relatively few instances of BSCRT that we were able to study were conducted 
by the grantee itself, rather than a separate service provider. Computer-aided 
instruction was common, but some programs used a workbook-exercise format. In 
either case, diagnostics conducted at the outset identified in which skills the participant 
was deficient, and instruction was oriented accordingly. Among the clear strengths of 
the examples we studied, all were open-entry and open-exit and emphasized self-paced 
learning geared to each student’s own areas of skill deficiency. 

Weaknesses included the fact that learning was usually decontextualized and 
opportunities for participatory learning or active learning methods were uncommon. 
Most importantly, due to limitations of funding, instructors in several of the programs 
were only available sporadically, so that supervision and mentoring were often not as 
intensive as would have been desirable. Such a loosely structured instructional format 
meant that only students who were strongly self-motivated tended to make steady 
progress. By contrast, in programs with greater structure and where instructors were 
continuously present and held students to high expectations, success rates were much 
more impressive. 
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In contrast to basic skills training, OSCRT was quite prevalent. Providers 
included state or tribal community colleges, private schools and colleges, and 
vocational-technical schools. Course offerings at these institutions were broad in scope, 
encompassing degree and certificate programs in myriad vocational fields, including 
Food Services, Child Care, Respiratory Therapy, Nursing, Graphic Arts, Welding, 
Business and Information Processing, and many others. Courses of study ranged from 
those lasting a few weeks to one or two years. Participants usually had a high school 
degree or GED at entrance, were generally required to have sound basic skills, and 
were typically highly motivated; thus they could be considered the cream of the JTPA 
crop. By implication, few provisions were in place for providing access to OSCRT for 
those who were less well prepared. 

The examples we studied were almost uniformly of high quality. Learning 
objectives were usually well specified, and curricula were practical, applicable, and 
focused on the development of specific skills needed in a work setting. Subsidiary 
instruction often focused on important ancillary skills, such as leadership, diligence, 
communication and problem-solving skills, and goal setting, and Native American 
cultural elements were also sometimes a part of the instruction. Active teaching 
methods and contextual learning were predominant; thus instruction was “hands on” 
and students were “learning by doing,” engaged in projects related to real-world or 
simulated work settings, and worked in teams with their classmates. Instructors strive 
to remain current, and designed their courses with employers’ needs in mind. They 
also exhibited high levels of professionalism and competence, and, whether Indian or 
non- Indian, showed dedication, sensitivity, and a strong caring attitude to program 
participants. 

On-the-job Training 

On-the-job training was used by 14 of our 23 case study grantees, but was used 
extensively by only a few programs. Surprisingly, tribal grantees were somewhat more 
likely to make use of OJT than non-tribal programs, and, due to the dearth of private- 
sector employers, job assignments were often with tribal or other public agencies. 

Grantees identified several advantages of OJT, including its potential for leading 
to long-term employment, providing participants with solid job skills that would be 
transferable to other work settings, and supporting community economic development 
in Indian-operated businesses and agencies. However, grantees also noted what they 
felt to be OJT’s drawbacks, including its relatively high cost and the potential for 
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employer abuse, which caused them to limit the extent to which it was used. 

Moreover, in some communities it was difficult to recruit employers to participate, 
either because there were few employers in the community to begin with or because of 
overt or covert racism. Finally, tribal grantees often needed to obtain approval for OJT 
contracts from tribal officials, and this sometimes led to lengthy delays. 

Among grantees who used OJTs, two approaches were common. In the first 
approach, grantees developed individual OJT assignments with diverse employers, 
typically for program participants who already had basic work maturity skills, adequate 
basic skills, and usually some relevant rudimentary job skills and appropriate career 
interests. In such cases, the specifics of the OJT arrangement needed to be 
individualized to the employer’s and participant’s circumstances, including the length of 
the contract, the participants’ working hours, and the training plan; doing so proved to 
be extremely time consuming but, when well done, led to a high quality training and 
work assignment. 

An alternative approach that some grantees used was to develop multiple OJTs 
with a few employers or with targeted industries, in some cases to support local 
economic development initiatives or tribal enterprises (e.g., tourism or gaming). These 
might take the form of simultaneous group OJT assignments or a steady sequence of 
individual participants being assigned with the same small number of employers over 
time. In any case, grantees using this strategy found that a single training plan could be 
developed as a “template,” into which, with only small adaptations, the names of 
individual participants could be inserted in turn. In this way, substantial efficiencies 
could be realized. Because multiple OJTs were being developed from the same model, 
great care was usually taken in ensuring that the template was well developed. At the 
same time, grantees needed to guard against being overly formulaic or failing to adapt 
the work assignment to the needs of the individual participants. 

Regardless of which approach was being used, high quality OJTs required that 
grantees work to develop good relationships with employers, be sure that participants 
were assigned to positions that matched their interests and abilities, work with 
employers and participants to develop meaningful training plans, adjust the length of 
the OJT contract to the amount of training to be provided, and monitor worksites to be 
certain that high quality training actually occurred and that participants were adequately 
supervised and transitioned to permanent employment. When evaluated against these 
criteria, we found that the quality of the OJTs we studied was mixed. In many cases, 
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the OJTs were carefully selected and monitored and the grantee took great pains in 
ensuring that quality training was being provided in skills in which the participant was 
deficient. In other cases, however, the work assignment was poorly matched to the 
participants’ interests and skills and the training plan amounted to little more than a job 
description. 

Work Experience and Community Service Employment 

Work Experience (WEX) and Community Service Employment (CSE) provide 
opportunities for participants to acquire hands-on work experience and skills training, 
while also gaining immediate income. In general, WEX is used for entry-level 
workers, while CSE is used for those with more experience. In some cases, for 
example, participants start out in WEX but switch to CSE when they have reached 
WEX’s 26-week limit. Beyond these differences, there is substantial overlap between 
these activities with respect to the types of positions in which participants are placed 
and the training and community objectives associated with the work assignments. 

WEX is among the most commonly used service options and is used by grantees 
in very diverse ways. Its objectives include providing participants with an introduction 
to the work world or a specific career, teaching occupational skills, providing 
immediate income, boosting self-esteem, and facilitating the transition to placement in 
unsubsidized employment. In some cases, it is used as a stand-alone activity, while 
elsewhere it is used in conjunction (either sequentially or concurrently) with another 
service activity, such as classroom training. In some instances, it is used as a short- 
term or stopgap measure to provide participants with needed income while they 
undergo job search, complete training, or gain an exposure to the world of work; in 
other cases, it is used, along with CSE, to provide long-term work assignments for 
individuals, often up to the allowable time and budget limits. This very diversity is part 
of the reason for the high frequency of use of subsidized employment in the INA 
programs we studied. Additionally, this activity often makes the most sense on 
reservations marked by an absence of economic development and with limited 
opportunities for unsubsidized employment, where direct placements and OJTs are 
impractical and classroom training cannot be very easily motivated. 

In addition to meeting the needs of the participants for income and training, WEX 
and CSE are used very explicitly to advance community and tribal interests, especially 
in the face of such economic conditions. Very commonly, tribal agencies come to 
depend on subsidized labor provided by the JTPA program to meet their staffing needs, 
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including running tribal government offices, Indian Health Services, tribally run day 
care centers, and even the JTPA office itself, among others. Subsidized employment 
thus becomes a way in which the tribe could provide valuable social services to the 
community that it often could not otherwise afford. In some cases, too, the use of 
JTPA-funded subsidized employment had become so institutionalized that it was used to 
screen and train potential new hires when job vacancies for unsubsidized employment in 
tribal agencies occurred. Thus, tribal members who aspired to full-time permanent 
employment with the tribe knew that the likeliest bet was to first become a JTPA 
participant and accept a WEX or CSE assignment. 

Given the great variety of ways in which subsidized employment is used and the 
myriad individual and community objectives that grantees needed to balance, it is 
difficult to unambiguously identify criteria for high quality program designs. However, 
grantees did not always ensure that participants’ training objectives were well 
articulated or that they were well matched for the positions in which they were placed, 
even for WEX and CSE assignments that were of substantial duration. Instead, 
meeting the community’s need for subsidized employment and the participants’ needs 
for immediate income often held sway. In any case, participants clearly appreciated the 
opportunity to work in an Indian-affiliated agency for and with their own people. 

Training Assistance and Placement and Post-Termination Services 

Services identified under this heading include career counseling, job development, 
job search assistance, instruction in pre-employment and work maturity skills, job 
referral and placement services, and vocational exploration. All grantees provided 
these services to participants, either as stand-alone activities or in conjunction with 
other training or subsidized employment, and most participants could be said to have 
received training assistance (TA) of some sort. 

Because TA was such a pervasive activity, facilitating access for participants who 
were geographically dispersed was a continual challenge. Grantees met this challenge 
partly by outstationing staff at other tribal agencies; however, due to what was 
sometimes a reluctance to work too closely with non-Indian agencies, linkages with the 
local workforce development systems (e.g., One Stop) were rarely very well 
developed. An additional strategy that is just starting to emerge is promoting self- 
access services, where (for example) jobs can be searched for electronically; grantees’ 
hesitancy in embracing this approach, however, stems from their reluctance to give up 
the personalized approach to services that is so much a hallmark of the INA program. 
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Providing TA in a way that reflects the cultural context and Native American 
values and customs was a particular strength of the INA programs that we studied. 
Along these lines, grantees adopted a holistic approach to services and typically placed 
the family and community at the center of services, rather than just the individual 
participant. Thus, grantees attended to participants’ entire physical and emotional well- 
being, and not just their needs for career guidance and job placement assistance. 
Similarly, counseling included attention to the well-being of the family as a whole, 
including health care needs, emotional attachments, and the like. 

As a demonstration of cultural sensitivity, grantees also made a particular effort 
to cultivate listing of jobs in which Native American participants could feel 
comfortable. For this reason, grantees often emphasized making job referrals in 
Indian-owned or operated businesses or agencies. Being referred for employment to a 
firm that provided a comfortable and welcoming environment was something in which 
many participants clearly placed great value. At the same time, the focus on providing 
job referrals sometimes caused programs to neglect developing participants’ job search 
skill, which would promote their longer-term self-sufficiency. Moreover, the emphasis 
on referrals in Native American businesses sometimes came at the expense of referrals 
with non-Native American firms that might have paid more or were more conveniently 
located. Thus, grantees need to balance participants’ desire for placements in 
environments in which they can feel more comfortable with their desire for placements 
in jobs that pay well, offer the opportunity for advancement, have favorable fringe 
benefits, and match their career interests. Helping participants adapt to non-Indian 
work settings, and helping non-Indian employers be more sensitive to the needs of 
Indian workers, were strategies that some grantees used to better balance these 
objectives. 

Factors Influencing Service Designs 

A number of factors again and again emerged as key influences on grantees’ 
service designs. These factors include the overall inadequacy of funding, which 
severely limits the number of persons who can be served, from among the much larger 
number who are program eligible and in need of assistance. Lack of funding also 
constrains grantees’ decisions about the types of services that participants are provided. 
Along these lines, recent reductions in JTPA funding have caused many grantees to 
shift towards lower cost service alternatives, such as Training Assistance, and away 
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from Classroom Training and Work Experience; it has also caused them to limit the 
amounts of supportive services that are provided. 

The absence of job opportunities in many Indian communities was another key 
constraint on service design and delivery. Without a reasonable prospect for placing 
many participants in unsubsidized employment, grantees needed to make difficult 
decisions about how best to use their employment and training funds, and often found 
that it was difficult to convince participants to undertake long-term skills training. 
Compounding the dilemma, clients’ consistent and clear preferences for remaining 
within the Indian or tribal community further limited choices. Fortunately, in some 
tribal settings, the introduction of gaming or other successful economic development 
efforts have served to open up additional employment opportunities. 

In light of these and other constraints, grantees were overall fairly effective. 
Among the most notable elements was their holistic approach to services and the 
overwhelming commitment and dedication of program staff. 

Summary Observations 

At the level of federal oversight and policy, the Department of Labor has over the 
last several years made enormous strides in forging a strong partnership with the 
grantee community. Although the degree of cooperation evidenced in the DOL-grantee 
relationship has waxed and waned over the past decades, a fundamental underlying 
sense of mistrust, even antagonism, has characterized their interactions. DINAP’s 
outgoing chief, Mr. Thomas Dowd, who is himself a former grantee director, has 
worked hard to dispel suspicion and forge a constructive dialogue. Although some 
grantees remain wary, the last several years have as a consequence witnessed a major 
transformation in the way DINAP is perceived. To an extent unforeseen only a few 
years ago, grantees now feel a sense of ownership towards “their” program and are 
working cooperatively with DINAP on all matters affecting program policy. Based on 
this foundation, the INA program is now well positioned to develop a sound future 
under the new Workforce Investment Act. 

In terms of service design and delivery, this future will doubtless reflect the 
unique role that INA programs play in the employment and training system and the 
many special strengths that were in evidence among the programs we studied. To begin 
with, INA grantees epitomize the holistic approach to services. Programs we studied 
were inordinately conscious of viewing clients as whole people rather than as instances 
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of symptoms to be treated. This approach often extended to a consideration of the 
needs of other members of the participant’s family and the ways in which family 
dynamics needed to be taken into account to promote the individual’s own success. 

The notion of holistic services was expanded as well to the conceptualization of 
the grantee community as a constituency in its own right and as an important 
beneficiary of the services that were being provided to individuals. Thus, grantees 
sometimes needed to strike a balance between deciding how to meet the needs of 
individuals for employment and training assistance while ensuring that the community’s 
own larger interests were being advanced as well. The ways in which Work 
Experience and Community Service Employment are used clearly exemplify this. 

The importance of community surfaced in another way as well, particularly for 
programs operating away from a tribal setting. In such environments, participants (and 
potential participants) often feel estranged and alienated from the mainstream culture 
and look to the organization operating the Section 401 grant as providing a needed 
sense of belonging. Moreover, participants take great comfort in knowing that their 
needs for assistance can be met by program staff that share a common ethnic identity as 
Native Americans and are sensitive to tribal cultures. So important is this shared 
identity that many participants would not avail themselves of similar services that might 
be made available from a non-Native American organization. 

For their part, grantees, again particularly those in non-tribal settings, become a 
focal point for community identity. As such— and, again, reflecting their holistic 
approach to services and their focus on meeting individual as well as community 
needs— they use funds from a variety of sources to organize cultural activities and 
events, promote native arts and crafts, run programs for senior citizens, operate Food 
Banks for the needy, publish community newsletters, and the like. The strong sense of 
community engendered by these efforts is largely irreplaceable and constitutes one of 
the strongest aspects of the uniqueness of the INA program. Although many of these 
subsidiary efforts rely on non-401 funds and a strong spirit of volunteerism, the JTPA 
allocation constitutes the bedrock of grantees’ funding and thus must be viewed as the 
foundation of these myriad efforts. 

Grantees operating in tribal settings, by contrast, to a lesser degree serve as the 
community’s focal point, because other tribal agencies and institutions can play this 
role. Instead, the JTPA program must be viewed as constituting an integral cog in a 
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wider network of services developed from other funding streams, used in tandem to 
meet the needs of individuals and the community. 

The above considerations suggest the importance of evaluating the INA program 
in context. The importance of context is suggested as well by the fact that grantees are 
operating their programs in extraordinarily diverse and often extremely difficult 
circumstances, typically with very limited funding. For example, service areas in many 
instances are marked by high rates of joblessness, physical isolation, and extreme 
economic deprivation. Fashioning an employment and training program in such 
circumstances poses special challenges and gives rise to difficult decisions regarding 
how a job training program should focus its energies in the face of a dearth of 
unsubsidized job opportunities of any kind. 

The importance of context suggests that it is very difficult to analyze specific 
aspects the Section 401 program (such as specific training activities) in isolation. In a 
real sense, in fact, the value of the program is much greater than the sum of its 
component parts. Thus, specific activities or services, when viewed in isolation, often 
seem unexceptional, but the INA programs taken as a whole play a critical role in 
promoting the vitality and well being of Indian and Native American individuals and 
communities. 

Recommendations 

Based on our intensive examination, we are convinced of the critical role that the 
INA program plays for the people and communities that they serve. We applaud the 
efforts of those grantees who design and deliver quality services in the face of 
inordinately difficult challenges. 

At the same time, we have drawn on our study’s findings to formulate a number 
of recommendations, which we have formulated at the level of federal policy and 
practice and the level of the grantee service design and delivery. 

Federal Policy and Practice 

1 . DOL must take seriously, as it has begun to do over the last several years, its 
obligation to work in partnership with the grantee community. Recent initiatives 
undertaken by the Division of Indian and Native American Programs (DINAP) , in 
conjunction with the Division of Performance Management’s Office of Policy and 
Research (OPR) , have been successful in enabling grantees to feel ownership of 
their program. DOL and grantees are now negotiating in a spirit of openness and 
cooperation to make important decisions about the program’s operation and future 
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direction. Given that Mr. Dowd recently stepped aside as the Chief of DINAP, 
DOL needs to ensure that his successor is as committed to dialogue and partnership 
as he was. The initiatives that Dowd began should not be allowed to falter. 

2. DOL needs to ensure that all grantees have the opportunity to participate in the 
partnership initiative. Results from our Administrator Survey suggest that some 
grantees, while applauding the partnership initiative, feel that they have not had full 
opportunity to participate. Although Work Groups and advisory bodies can 
understandably include just handfuls of members, DOL should ensure that all 
grantees have ample opportunity to express their opinions on matters affecting the 
program. Where it would not be too disruptive, Work Group or partnership bodies 
should include provisions for rotating memberships, and efforts need to be made to 
see that all grantees, including tribal and non-tribal programs and those that are 
large and small, are well represented. DOL’s recent initiative to promote Internet 
access for all grantees may provide an additional vehicle for giving all grantees the 
chance to participate in a dialogue. 

3. DOL needs to work with grantees to ensure that new regulations for the Section 401 
program are clear, concise, and grant ample flexibility to grantees to design and 
operate their programs in accordance with the needs of their communities. The 
recently enacted Workforce Investment Act (WIA) of 1998 supplants JTPA, but still 
allows a provision for a separate, national Indian and Native American Program. 
Doubtless, DINAP will need to craft new regulations for the program under WIA. 

In keeping with recommendations expressed above, new regulations should be 
developed with the full cooperation of the grantee partners. In recognition of the 
great diversity of contexts within which the INA program operates, we recommend 
that these regulations permit ample flexibility to grantees for designing programs 
that are responsive to local needs. Overly restrictive provisions on service designs 
that are not statutorily mandated should be avoided. Any new regulations, as well 
as all other DOL issuances (e.g., DINAP Bulletins), should also be written with any 
eye to simplicity and clarity. 

4 . The new regulations need to permit WIA funds to be used to promote diverse 
individual and community needs, including economic development if possible. Lack 
of adequate job opportunities and weak economies are among the biggest obstacles 
Section 401 grantees face in accomplishing their program objectives. Given these 
circumstances, concerted efforts should be made to spur economic development in 
Native American communities. The JTPA Amendments of 1992 placed severe 
restrictions on the use of JTPA funds for economic development activities, 
effectively eliminating a promising Community Benefits Projects (CBP) initiative 
that had previously been permitted in the INA program. 1 To the extent that it is 
allowable under WIA, we encourage DOL to consider reinstating provisions for 



1 The provisions of CBP allowed grantees to use a limited amount of their JTPA funds to engage 
in workforce development and training in the context of promoting local economic development. 
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Community Benefits Projects, or similar initiatives to allow WIA funds to be used 
to promote, or in conjunction with, economic development efforts. 

5. DOL should forge ahead with plans to revamp the performance standards system for 
the IN A program, to reflect the wide diversity of grantees' circumstances and 
accomplishments. At the same time, it must be careful to ensure adequate 
accountability at the national level. Grantees are clearly ambivalent about the 
current performance measurement system. Although the current measures are 
generally perceived as being reasonable and fair, many grantees also feel these 
measures do not fully reflect their programs’ chief accomplishments. Thus, DOL’s 
recent efforts to work in partnership with the grantee community to develop a 
revised performance-standards system seem wholly appropriate. The recent 
enactment of the Workforce Investment Act should not delay these efforts. 

The twin themes of this new legislation are, on the one hand, to devolve 
substantial authority for decision-making to local programs, and, on the other, to 
ensure that local programs are held strictly accountable for their performance. In 
keeping with these themes, DOL needs to ensure that new performance measures 
(as well as program regulations in general) impart substantial flexibility, while at 
the same time ensuring that adequate accountability mechanisms are in place. This 
will require that any new performance-standards system provide meaningful and 
substantial measures of accountability. Additionally, the new measures need to be 
supported by an adequate reporting vehicle that includes clear definitions of key 
terms and the valid and reliable measurement and reporting of key concepts. 

6. DOL needs to ensure that additional attention is paid to the needs of grantees for 
technical assistance and training, especially those who are new Directors, and that 
grantees have adequate opportunity to engage in dialogue with their peers and 
DINAP's Federal Representatives. Any new flexibility imparted under WIA implies 
as well that grantees be provided with sufficient opportunities to learn how to take 
advantage of that flexibility by designing effective and innovative services. Along 
these lines, although the technical assistance and training (TAT) that has been 
provided heretofore has been adequate overall, certain segments of the grantee 
population, especially new Directors, need additional assistance. 

In general, capacity building also needs to be promoted, and grantee staff 
should have the opportunity to build their skills in all areas. Assessment is a 
particular area in which programs might benefit from additional capacity in the 
years ahead. Participants will increasingly need to improve their skills to compete in 
the labor market; schools’ and employers’ expectations and requirements have risen 
over the past decade and will continue to place more importance on excellent 
academic and workplace skills into the next century. In this context, a careful 
assessment should be the foundation upon which participants’ services are based. 
Without a thorough knowledge of participants’ interests and abilities, service plans 
can be made based only on a general feeling for the clients’ capabilities and long- 
term needs, and thus risk missing the mark. Given this, staff will need to have 
training on how best to access assessment results and how to interpret them. 



Grantees could also benefit from additional efforts to learn about what their 
colleagues are doing, including through peer-to-peer exchanges. DOL should also 
continue its recent efforts to ensure that grantees have adequate opportunity to meet 
individually with DINAP’s Federal Representatives. 

Grantee Service Design and Delivery 

7. To ensure a more equitable access to services, grantees should avoid an over- 
reliance on word-of-mouth referrals. For the same reason, they need to make 
provisions for reaching potential applicants throughout their service areas, through 
outstationing staff and forging partnerships with other social service agencies. 
Outreach and recruitment efforts currently being undertaken are clearly adequate for 
ensuring a constant flow of participants. However, some grantees rely almost 
exclusively on indirect recruitment methods, especially word-of-mouth referrals, 
and this has sometimes made it difficult for potential applicants who are less well 
connected in the community to be made aware of services. Thus, some participants 
with whom we spoke, who eventually heard about the program through a friend or 
relative, told us that they wished they had learned of the program’s existence years 
earlier. 

An additional difficulty that grantees experience in ensuring an equitable access 
to services is in reaching potential participants throughout the entirety of the 
grantees’ service areas. Achieving this objective is an especially difficult challenge 
for those grantees serving physically vast territories but who find it financially 
infeasible to establish separate field offices, due to their very limited funding. In 
such cases, grantees can follow the lead of many of their colleagues who outstation 
staff, use roving recruiters, and develop effective joint referral linkages with other 
social service and tribal programs. As the new WIA legislation takes hold, linkages 
with the nation’s emerging One-Stop systems can be an especially promising 
practice that is rife with opportunity, as a few grantees have already demonstrated. 
Thus far such linkages appear to be much underutilized. 

8. While grantees need to attend to participants’ immediate needs for income and 
employment, they need to address clients’ longer-term needs as well. We found that 
clients often had strong opinions about what services they wanted and needed in the 
short-run, typically including obtaining immediate employment (subsidized or 
unsubsidized), driven by their need for immediate income. Consequently, Section 
401 grantees often emphasized addressing participants’ shorter-term needs, 
sometimes to the neglect of their longer term needs, and resulting in quick-fix 
solutions and a “revolving door” approach to services. We recognize that grantees 
often find themselves severely constrained by limited budgets and other factors. We 
also appreciate that they must be responsive to the expressed preferences of their 
clients. To the fullest extent possible, however, grantees should promote long-term 
solutions and structure service strategies to advance participants’ longer-term 
interests while attempting to address their needs in the short run. Suggestions for 
how to do so are embedded in some of the recommendations that follow. 
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9. Grantees who are not located in reasonable proximity to service providers for 
Classroom Training should consider utilizing distance learning or alternative 
delivery vehicles. Extreme physical isolation is a major impediment to making the 
full-range of classroom training services available to participants, at least among 
some grantees. Alternative or innovative service delivery vehicles can be pursued 
in these circumstances, including distance learning or other on-line classroom 
services. The greater access to technology among both grantees and participants 
will make this more and more feasible. 

10. Grantees need to ensure that On-the-job Training is accompanied by a clear 
training plan, specifying the specific skills the participant is expected to learn, and 
in fact provides training opportunities commensurate with the employers ’ wage 
reimbursements. On-the-job training (OJT) has great potential as a service activity 
in imparting meaningful skill gains and leading to permanent employment, and to 
this degree can clearly be said to address participants’ longer-term, as well as short- 
term, needs. However, to realize this potential to the fullest, grantees must ensure 
that the participant’s training plan is clearly specified, that the work assignment 
imparts meaningful skills that are transferable across employers, and that the 
employer provides adequate supervision and mentoring. Moreover, grantees should 
have the expectation that employers will continue to hire the trainee once the 
training period has ended, and limit their involvement with employers who fail to 
do so. Many of the OJTs we studied exemplified these characteristics, but many 
others did not. Grantees should ensure that all OJTs attempt to promote high- 
quality design principles. 

1 1 . Work Experience assignments of substantial duration should also be structured to 
provide real and meaningful training. Work Experience (WEX) serves very diverse 
objectives and is used by grantees in very different ways. In some cases, WEX is 
deliberately and appropriately structured as a short-term or stopgap work 
assignment, as when participants are given short-term WEX positions while 
undergoing job search or as a way of providing exposure to the world of work and 
boosting self-esteem. Very often, however, WEX assignments are of substantial 
duration (e.g., up to the 6-month limit). In these cases, grantees should ensure that 
training objectives are clearly specified and the participants’ are learning valuable 
occupational skills. By doing so, grantees can again promote participants’ long- 
term needs for quality skill development, as well as their short-term need for 
immediate income. 

12. For participants interested simply in direct placement assistance, grantees need to 
ensure that they provide not only job referrals, but also that they build participants’ 
job search skills. Section 401 grantees very appropriately often provide direct 
placement assistance. In doing so, they should avoid simply giving job referrals, 
especially when participants seem to lack good job search skills. In such cases, 
grantees that provide training in job search skills boost participants’ self sufficiency 
and empower them to seek and find their own jobs in the future, minimizing their 
subsequent need for the program’s assistance. 
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I. INTRODUCTION 



This report presents the findings from a 30-month evaluation of the JTPA 
Title IV Section 401 Indian and Native American (INA) Program. The study has been 
conducted by Social Policy Research Associates (SPR) and its partner, American Indian 
Research and Development (AIRD), and was funded by the U.S. Department of Labor 
(DOL) as part of its broad oversight responsibility for Title IV programs and as a 
reflection of its commitment to fostering continuous program improvement. 

The evaluation consisted of two major components: a qualitative process study 
and a quantitative analysis of survey and other program data. The qualitative 
component, on which the bulk of this report is based, draws on site visits made by 
research staff to 23 randomly selected INA grantees. During these visits, which 
typically lasted 2 or 3 days each, field staff held discussions with program 
administrators and staff, classroom instructors and worksite supervisors, and program 
participants and recent terminees, and they observed instances of training and other 
services being conducted. The intent of these visits was in no way to monitor the 
grantees’ compliance with regulatory requirements, but was instead to help us 
understand the goals and objectives programs have established for themselves, the 
service design decisions that resulted, and the overall quality and effectiveness of those 
designs in meeting the needs of participants and communities. 

Complementing these site visits was the quantitative analysis of data from a 
variety of sources. Chief among these was a survey administered to all Section 401 
grantees, intended to be completed by each program’s Executive Director. The survey 
asked respondents to assess the helpfulness of the support and guidance provided to 
their programs by DOL, to characterize their program’s scope and objectives, and to 
describe the service planning process and the most important barriers to achieving 
successful outcomes for participants. A major additional data source was the Annual 
Status Report, the chief reporting vehicle in the INA program through which grantees 
provide tallies of the persons they serve, their characteristics, and their outcomes. 

Organization of The Report 

The remainder of this chapter provides a brief overview of the INA program, 
focusing on some of the challenges in serving Indians and Native Americans and the 
ways in which the program has been designed to meet these challenges. By way of 
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establishing the context, it also discusses the unique relationship between the Indian and 
Native American community and the federal government. The chapter then concludes 
with an overview of the conceptual model that has guided the evaluation effort, both 
presenting a heuristic model of service design and delivery and identifying key criteria 
by which to judge the quality of services. 

Chapter II next details the study design, including the manner in which the case 
study sites were drawn, the protocols for the site visits, the process for designing and 
administering the survey, and our procedures for soliciting input on the study design 
and methods from grantee representatives. 

Chapter III represents the first analysis chapter, and is drawn exclusively from 
the grantee survey. This chapter presents tallies from the questionnaire items and 
quotes extensively from the numerous written comments provided by grantees, on 
topics that include the respondents’ appraisal of the adequacy of the guidance they are 
receiving from DOL as well as their identification of key elements of their service 
designs. 

Chapter IV, which, like most of the remaining chapters, draws primarily on the 
results from the site visits, takes a broad-brush look at grantee organizations and 
service strategies, and includes sections on goals and objectives, funding, staffing, and 
service emphases. 

The next chapter, Chapter V, begins the first of a sequence of chapters looking 
in-depth at specific elements of service delivery. This chapter concentrates on outreach 
and recruitment, the processes by which participants’ needs are assessed and their 
service strategies are developed, and the use of supportive services and stipends to 
support participants through training. 

The next four chapters focus in turn on basic skills and occupational skills 
classroom training, on-the-job training, work experience and community service 
employment, and training assistance and placement services. Each of these chapters 
has a similar organization and provides a discussion of the ways in which each service 
or training type is used and an assessment of the quality of services that are provided. 

Finally, Chapter X presents an overview of factors influencing program design 
decisions, and Chapter XI concludes with an assessment of the programs’ overall 
effectiveness and makes some policy recommendations. 



Background 



Concerned Americans should decry statistics that describe the deplorable 
socioeconomic, health, and employment conditions of many residents across Indian 
country. Figures from the 1990 Census and elsewhere show that, by virtually all 
indicators, many Indian people on and off the reservation suffer from extreme poverty 
and extraordinarily high rates of unemployment. For example, about 30% of American 
Indian households, or over twice the national average, have incomes that place them in 
poverty, and Indian poverty rates in some states and reservations are much higher still. 
Linked to these poverty rates are high rates of joblessness. In 1995, as estimated by 
the Bureau of Indian Affairs, the national unemployment rate for Indians in BIA service 
areas was almost 35%, compared with a national U.S. rate of 5.6%. On some 
reservations, unemployment has reached catastrophic proportions: Cheyenne River 
(77%), Fort Totten (59%), Santee Sioux (74%), Fort Sill Apache (80%), and Taos 
Pueblo (45%). Even off-reservation Indians generally fare poorly in the job market, 
with unemployment rates well above the national average. 

Of course, many tribal communities and Indian people have made tremendous 
economic progress and achieved high levels of socioeconomic success. But the general 
lack of a strong economic base in many Indian and Native American communities is 
indeed persistent and pervasive. Moreover, weak economies have given rise to 
widespread social and health problems. For example, community health is generally 
poor and Indian life expectancies are substantially lower than in the U.S. population at 
large. 

Addressing the Need for Employment and Training Services: The INA 

Section 401 Program 

The challenge in redressing these circumstances in light of the very unique 
historical and cultural context of Native American communities has provided ample 
justification for a specially-targeted program under JTPA. With these circumstances as 
a backdrop, Congress established Section 401 of JTPA as a special program providing 
employment and training services to Indians and Native Americans. Currently funded 
at about $52 million per year, the program provides services through a network of 
approximately 175 grantees (with the precise number varying somewhat from year to 
year). About two-thirds of the grants are run by tribal governments, either singly or as 
a combination of separate tribes who have entered into a consortium agreement to 
realize economies of scale. The remaining grantees are non-tribal programs that are 
generally run by community-based organizations or, sometimes, agents of state 
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governments. Each grantee serves a specified service area, but, in combination, the 
tribal and non-tribal programs provide services throughout all of the U.S. 

Funds are distributed to these grantees using an allocation formula that takes into 
account the concentration of Indians and Native Americans in poverty and their 
unemployment rates. DOL uses Census data as the most reliable available source on 
which to base these calculations. Resulting Section 401 grant awards range from a low 
of $20,000 annually to more than $7 million. Some grantees additionally receive JTPA 
Title II-B (Summer Youth) funds (available to tribal programs only), and they may 
apply for and receive funds from other sources as well, including the Departments of 
Health and Human Services or Housing and Urban Development, and the Bureau of 
Indian Affairs. Only the activities provided through the Section 401 program are the 
immediate focus of our evaluation, although grantees usually attempt to leverage their 
funds from multiple sources to support their program goals. 

Looking at the Section 401 activities, funds were used to provide services in PY 
96 to approximately 20,000 participants, about 16,000 of whom “terminated,” or left 
the program, generally after completing their service plan. 1 To meet the INA 
program’s eligibility criteria, participants must be “an Indian, Native Alaskan, or 
Native Hawaiian, as determined by the ... grantee, who is economically disadvantaged, 
or unemployed or underemployed...” or who is otherwise in need of skill upgrading or 
retraining (20 CFR Part §632.172). Thus, by going beyond the confines of economic 
disadvantage, the regulations give INA grantees substantially more discretion in who 
can be served than is typical elsewhere in the JTPA system. Moreover, and reflecting 
tribal grantees’ protection of their prerogatives, past efforts to amend the regulations by 
requiring targeting on harder-to-serve participants (i.e., those with substantial barriers 
to employment) were forcefully rebuffed. 

In common with most other JTPA programs, participants are offered access to a 
range of services that include classroom training, on-the-job training, work experience, 
job search assistance, and counseling and assessment services. Community service 
employment, unique to the INA program, is also offered. 



1 See Appendix B for basic information about the program, drawn from recent Annual Status 
Reports. 
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Serving the Unique Needs of Indians and Native Americans 

INA grantees face constraints common to programs operating under other titles of 
JTPA. For example, most programs find that federal allocations are woefully 
inadequate for serving all those eligible, and INA grantees are no exception. Indeed, 
employment and training programs targeted to the Indian and Native American 
population suffered massive cutbacks during the transition from CETA to JTPA and in 
recent years have seen their funds dwindle still further. Using available JTPA dollars 
as effectively as possible is, therefore, a central challenge, as are leveraging funds from 
other sources and developing coordinated program strategies. 

The INA grantees in common with programs operating under other JTPA titles 
also face the difficult task of addressing the multiple needs of program participants. 
Notwithstanding allegations by some that the JTPA system encourages “creaming,” or 
serving the most job-ready from among those eligible for services, many INA 
participants are clearly hard to serve and can have their labor market prospects 
significantly enhanced only with substantial skill upgrading. The further challenge is 
delivering these services while meeting the need many participants have for an 
immediate and steady stream of income to support themselves and their families. 

Beyond these considerations, however, INA grantees face unique constraints and 
challenges posed by the special needs of the clientele and the characteristics of their 
service areas. Some of these are summarized in Table 1-1. Leading the list is the 
virtual absence of private-sector job opportunities in many Indian and Native American 
communities. Of course, some INA programs operate in urban areas or in close 
proximity to active labor markets. But, as was mentioned earlier in this chapter, 
joblessness rates on many reservations reach levels that are unheard of in the rest of the 
nation. In some cases, there are literally no job opportunities of any kind except those 
few under the control of the tribal government. It is difficult to exaggerate the 
profound impact this circumstance has, not only to the lives of Indian people, but on 
the hard programmatic decisions that must by made by JTPA INA program directors. 

Closely linked with the lack of job opportunities is the difficulty of providing 
services in areas that are often physically remote and with poorly developed 
infrastructures. Not only do many Indian and Native American communities lack 
active labor markets, but they are geographically isolated and thus often not within a 
reasonable commuting distance of job opportunities or even job training service 
providers. 
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Table 11 

The Program Environment: 

What Makes the INA Program So Different? 


Context 


Unique Attributes of Grantees 


Absence of active labor markets in 
many areas. 


r®=Extreme disparity in the sizes of 
grantees. 


^Remote service areas with weak 
infrastructures in many areas. 

rsrPervasive discrimination against Native 
Americans. 


(^Diversity of grantee types 
— Tribal government programs 
— Tribal consortia 
— Non-tribal programs 


^Participants’ sense of alienation and 
isolation. 


(^■Extreme diversity in service area 
context 


^Established relationship between 
participants and staff. 




^Importance of cultural values. 
(^Importance of tribal sovereignty. 





Even programs operating in active urban economies must confront pervasive 
discrimination directed against Native Americans, which limits their employment and 
other opportunities. Additionally, participants in urban communities often experience a 
sense of alienation, caused by their immersion in an environment that seems to many to 
be foreign, unfriendly, or even hostile. In such circumstances, expanding participants’ 
access to economic opportunities while enhancing feelings of self-worth and providing 
a socio-psychological anchor must be important objectives. 

As part of establishing this anchor, program staff and participants build on 
established relationships that come from being members of the same community. 

Thus, in contrast to most other employment and training and social service programs, 
INA grantee staff and program participants typically have a long-standing person-to- 
person relationship; they and their family members interact directly in daily life: they 
shop at the same stores, attend the same church, attend community events together, etc. 

Consistent with this, an important challenge is providing for participants’ 
employment and training needs while being responsive to their culture, identity, and 



value systems as tribal members. Although Native American communities are 
themselves very diverse in their belief systems, customs, and traditions, INA grantees 
face the commonality of needing to be responsive to native cultural values. Even the 
JTPA legislation upholds this, by explicitly drawing attention to the need for programs 
to provide services “consistent with (the) goals and lifestyles” of native peoples 
(Section §401 .a. 3) . After a century of yo-yoing between an effort to uphold the 
government’s trust responsibility and abandoning this responsibility in the name of 
assimilation, federal policy, at least in principal, is now recognizing the right of Indian 
communities to maintain their uniqueness. 

Moreover, the grantee community is vociferous in its defense of the prerogatives 
of tribal sovereignty and Indian self-determination. This means, from their 
perspective, that they should be given wide latitude in making decisions about running 
their programs and that all DOL decisions should be issued only after extensive 
consultation. 

Exactly how INA grantees can best carry out the legislation’s mandate to provide 
services consistent with Native American goals and lifestyles is for each community to 
decide on its own. 

The Diversity of Grantee Types 

Despite the similar challenges they face, INA grantees are themselves 
extraordinarily diverse in their operational constraints and organizational contexts. As 
a result, very different problems loom large for grantees of different types. For 
example, as mentioned, grantees are enormously disparate with respect to their size. 
Very large and very small grantees face very different constraints and opportunities. 
Larger organizations, with JTPA funding in the millions, face the management 
challenges in coordinating a program with dozens of staff members located in multiple 
field offices spanning an enormous geographic expanse. Very small grantees, by 
contrast, may operate with just a few staff members who are responsible for all 
functions related to the JTPA program, including those involving program 
administration (e.g., overall program operations, fiscal controls, MIS) and training 
services (e.g., intake worker, counselor and case manager, trainer, job developer). In 
such cases, programs cannot realize economies of scale and it is difficult for them to 
develop a broad array of training options for participants. Leveraging and coordinating 
services become all the more important. Discussions initiated by DOL early in this 
decade encouraging those with very small allocations into consortia agreements to 
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realize economies of scale were hotly protested by the grantee community as an 
infringement on their sovereignty 

Grantees also face different challenges according to differences in their 
organization type. Many grantees are run by tribal governments, who need to manage 
their JTPA funds in the context of the tribal political structure and, oftentimes, in the 
maelstroms of tribal politics. Others are run by tribal consortia, who need to develop 
consensus and coordination with multiple tribes, each of which values their autonomy. 
Finally, others are non-tribal programs, who face the challenge of promoting their 
participants’ Indian identity in the face of an alienating culture. 

Programs, too, are diverse with respect to the socioeconomic contexts in which 
they operate. As was already mentioned, many programs operate on remote 
reservations or rural areas with a very limited private sector; developing job 
opportunities in such a setting as well as accessing appropriate training services for 
participants will be enormously challenging. Grantees operating in urban areas, by 
contrast, may find that job opportunities are more plentiful, but there is more of a need 
to attend to the participants’ cultural isolation and the discrimination they may 
encounter. 

These constraints and considerations pose formidable obstacles to effective 
program design and implementation. Understanding how grantees face these very 
different challenges has been an important topic for the evaluation. 

The Federal Role 

In contrast to most programs funded under JTPA, the IN A program is nationally 
administered. The Department of Labor’s Division of Indian and Native American 
Programs (DINAP) is charged with providing general program guidance and 
monitoring program performance. Among the ways it carries out these functions is by 
having a strong hand in crafting program regulations and by issuing periodic bulletins, 
which clarify matters of policy. It also participates in organizing regional and national 
technical assistance and training conferences, where grantee staff can receive further 
guidance on matters of policy as well as attend workshops to enhance their program 
management and service delivery skills. Further, DINAP Federal Representatives 
provide broad oversight of grantee operations and are available to confer with grantee 
staff on issues of concern. 
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Another way in which DOL shapes program priorities is by issuing performance 
standards. Required by the JTPA legislation, performance standards in the INA 
program are issued to each grantee and are outcome based, relating to the kinds of 
achievements attained by participants who received services and who terminate from 
the program. Currently, three performance standards are in use: 

• The entered employment rate (EER) , defined as the number of 
participants who enter unsubsidized employment at termination, as a 
percentage of all terminees. 

• The employability enhancement rate (EEN) , defined as the number of 
participants who attain an employability enhancement at termination, as 
a percentage of all terminees. Enhancements, which are taken to 
represent skill upgrades, are defined to occur for participants who: 

- Enter non-Section-401 training. 

- Return to school or remain in school. 

- Complete a major level of education. 

- Complete worksite training objectives. 

- Attain a basic or occupational skill proficiency. 

• The positive termination rate (PTR) , defined as the number of terminees 
who either enter unsubsidized employment or attain an enhancement (or 
both), as a percentage of all terminees. 

Grantees are required to meet their minimum standards on at least 2 of the 3 outcomes, 
or risk sanction. Minimum standards on EEN are set at a flat rate for all grantees, but 
those for EER and PTR are established for each grantee after taking into account the 
characteristics of its terminees and of its service area; in general, grantees serving a 
harder-to-serve clientele or operating in more economically depressed areas are 
assigned somewhat easier standards in recognition of the more difficult circumstances 
within which they are operating. 

In carrying out its responsibilities, DINAP, and DOL more generally, has not 
always had a harmonious relationship with the grantee community. Although the 
specific issues over which conflict has developed have changed, and the extent of the 
friction has waxed and waned, two key themes seem to continually resurface: 

• The nature of the consultative process. From the grantees’ point of 
view, DOL’s efforts in past years to engage in dialogue with the 
grantees before making policy decisions have been found sorely lacking. 
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• The appropriate balance between deference to grantees’ autonomy, on 
the one hand, and DOL’s oversight responsibilities on the other. 

Clearly, the government-to-government relations between Indian tribes 
and the federal government serve to uniquely frame the DOL-grantee 
relationship. 

Recognizing how much conflicts over these issues have hampered its effectiveness 
in working with the grantees, DINAP has recently undertaken a partnership initiative, 
which has entailed revisiting a number of policy-related issues— including the 
program’s regulations, reporting requirements, and performance standards— in close 
consultation with representatives from the grantee community. This initiative has gone 
a long way towards dispelling grantees’ feelings of mistrust that had accumulated over 
the years, and has been viewed as ushering in a new spirit of cooperation and 
collaboration between the department and the grantee community. 

The Special Relationship with the Federal Government 

The importance of cooperation and collaboration is underscored by the “special 
relationship” between American Indian tribes and the federal government. This context 
is so important to a proper understanding of the dynamics between DOL and the 
grantees that a brief discussion of it is in order here. The intent is not to provide a 
technical or comprehensive historical treatise, but a simple overview that puts in 
context the condition of Indian affairs in the United States and their relationship to the 
federal government, the states, and other tribes. 

United States taxpayers often assume that American Indian tribal people receive 
amenities above and beyond those provided to the “regular” American citizen. Some 
of these assumptions are that Indian citizens: (1) do not pay taxes, (2) all live on 
reservations, and (3) receive free health care, housing, education, or other social- 
economic-welfare services. These assumptions are clearly inaccurate. Moreover, 
whatever tribes and American Indian individuals receive has come at a high price. 

Indian-tribal and federal relations formally began with the drafting and signing of 
the U.S. Constitution. The drafters of the Constitution considered its design after 
learning of the Iroquois Confederacy, which had held the Iroquois people together for 
hundreds of years. Ben Franklin wrote in a letter to James Parker: 

It would be a very strange thing, if six Nations of ignorant savages should 

be capable of forming a Scheme for such a Union, and be able to execute it 

in such a Manner, as that it has subsisted Ages, and appears indissoluble; 

and yet that a like Union should be impractical for ten or a Dozen English 
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Colonies, to whom it is more necessary, and must be more advantageous; 
and who cannot be supposed to want an equal understanding of their 
interests. 2 

The drafters of the Constitution recognized the efficacy of a balance of powers 
and established Executive, Legislative, and Judicial branches of government, based on 
a like separation of powers contained in the Iroquois Confederacy. The original U.S. 
Constitution also twice mentions Indian people (the only mention of another race) , as in 
Article I, Section 2 “...excluding Indians not taxed” and Section 8 “...to regulate 
commerce with foreign nations, and among the several states, and with the Indian 
tribes.” 

The inclusion of Indian tribes in the same category as states implies a status equal 
to states, and that status is as a sovereign entity. “Indian governments, as sovereign 
nations existing within the boundaries of the United States, have a special legal and 
political relationship with the United States. This relationship is defined by treaties and 
special laws.’’ 3 The U.S. Constitution, once ratified, recognized Indian tribes as 
sovereign nations; however, most Indian tribes were sovereign before this recognition. 
Sovereignty may be defined as the supreme power that binds people together to form a 
nation and from which all other political powers are derived. This supreme power can 
only be defined by the individual sovereigns. Thus, according to this definition, Indian 
tribes exercised sovereignty hundreds or thousands of years prior to the United States 's 
recognizing their sovereignty via the Constitution. 

Treaties with Indian tribes (1778-1871) were the main vehicle used by the U.S. 
Government to accomplish several ends: peace, friendship, and trade with independent 
tribal nations. The U.S. Government assumed a “protective” role but tribes maintained 
civil and criminal jurisdiction over their lands and the right to self-government. Indian 
tribes were often forced to cede lands and agree to “peaceful co-existence,” and in turn 
received promises of protection, livestock, farm implements, education, and other 
provisions that in essence provided for the tribes’ well-being in perpetuity “...as long as 
the grass grows and water flows.” 



2 Benjamin Franklin to James Parker, March 20, 1751 , reprinted in The Papers of Benjamin 
Franklin, edited by L.W. Larabee, et al. , Vol. 4, pp. 117-21. New Haven, Conn.: Yale University 
Press, 1961. 

3 Lynn Kirk Kickingbird, Institution for Developmental Indian Law, Washington D.C. 
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Acts of Congress, such as the Northwest Ordinance of 1787, further affected 
tribes and reaffirmed the U.S. government’s recognition of tribal sovereignty. 

The Utmost good faith shall always be observed toward the Indians; their 
land and property shall never be taken from them without their consent; and 
their property, rights, and liberty never shall be invaded or disturbed unless 
in just and lawful wars authorized by Congress...” (Northwest Ordinance, 

1787). 

The treaties signed in “utmost good faith” with tribes guaranteed many things; in 
effect, the U.S. government promised to provide health, education and welfare in 
perpetuity. The tragedy of the treaty-making is that each of the treaties was broken. 
Moreover, treaties placed Indian tribes on reservations, where they were expected to 
become farmers, regardless of their previous lifestyles as hunters or gatherers, and 
notwithstanding that the reservations were more often than not in arid, less than 
productive (for farming) land. As a consequence, many tribes were decimated or 
disrupted and today suffer the consequences, including the lowest income levels, the 
lowest educational achievement levels (highest dropout rates), the highest suicide rates, 
the poorest housing conditions, the highest adult and infant mortality, and the highest 
unemployment rates. 

The U.S. federal government, in recognition of the special status of Indian tribes, 
has passed laws that acknowledge the special rights of Indian tribes and people apart 
and distinct from other American citizens. Some significant laws specific to Indian 
tribes are: 



• 1778 - First treaty between the United States and Indian Nations (treaty 
with the Delawares) . 

• 1787 - Northwest Ordinance - to assure Indian tribal sovereignty and 
consultation. 

• 1790 - Intercourse Act - to strengthen enforcement and prosecution of 
treaty violations by white traders. 

• 1818 - Civilization Act - to provide $10,000 to convert Indians from 
hunters to agriculturists. 

• 1831 - Indian Removal Act - to remove Cherokee, Choctaw, Creek, 
Chickasaw, and Seminole to Oklahoma from their traditional homelands 
in North Carolina, South Carolina, Georgia, Florida, and Tennessee. A 
proposal was developed to form an Indian state. 

• 1871 - End of treaty making period. 
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• 1887 - General Allotment Act - Federal policy to break up 
reservations. 

• 1 892 - Commissioner authorized by Congress to withhold food and 
services to enforce mandatory school attendance, as a means to 
assimilate Indians into larger American society. 

• 1924 - Congress, via the Indian Citizenship Act, grants U.S. citizenship 
to all Indians. Indian people are tribal citizens as well as U.S. citizens. 

• 1928 - Merman Report was issued. The First comprehensive study of 
the Indian “problem.” Basic conclusions were: (1) Indians were getting 
poor services, in health and education, from public officials charged 
with meeting these needs, and (2) Indians were being excluded from the 
management of their own affairs. 

• 1934 - Indian Reorganization Act - Congressional attempt to restore 
tribal governments. Introduced democratically elected officials and 
tribal constitutions. 

• 1934 - Congress passes the Johnson-0 ’Malley Act, which authorized 
contracts for welfare and educational services to entice states, 
territories, and public schools to assume more responsibility for 
providing elementary and secondary education, medical attention, 
agricultural assistance, and social welfare for Indian children and adults 
who reside on Indian reservation lands. 

• 1952 - Congress passes a program to relocate Indians from the 
reservations. This program subsidized Indian individuals to move to 
urban areas (Los Angeles, Oakland, Kansas City, Dallas, Seattle, 
Denver, Chicago, etc.) and provided subsistence (food, lodging, and 
health) for a period of time. Relocated Indians were expected to 
become “productive citizens” by joining the work force, and some did, 
resulting in inter-tribal urban Indian populations; others did not 
“succeed” and returned home to the reservation. 

• 1953 - Public Law 83-280, Indian Termination Act. The federal 
government terminated all services to several tribes, which in effect 
brought almost total destruction to those tribes and has taken them 40 
years to recover. 

• 1968 - Kennedy Report is issued, Indian Education, A National 
Tragedy, A National Challenge: This report described the abysmal 
condition of Indian education and Indian affairs across the U.S. and was 
the basis for the Indian Education Act of 1972. 

• 1970 - President Nixon issues an Indian policy statement: 

“It is long past time that the Indian policies of the Federal government 
began to recognize and build upon the capacities and insights of the 
Indian people. Both as a matter of justice and as a matter of 
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enlightened social policy, we must begin to act on the basis of what the 
Indians themselves have long been telling us. The time has come to 
break decisively with the past and to create the conditions for a new era 
in which the Indian future is determined by Indian acts and Indian 
decisions. 4 

• 1972 - Indian Education Act passed. Reformed the way Indian 
Education was being conducted by amending the Elementary and 
Secondary Assistance Act to provide funds for Indian education in 
schools they attend, mainly public schools, “to meet the academic and 
cultural related educational needs.” 

• 1973 - Congress passes the Comprehensive Employment Training Act. 
Title III, Section 302 established a special program for American 
Indians and Alaska Natives to provide training and employment to 
individuals and advance the social and economic development of Indian 
communities "... consistent with their goals and life styles.” 

• 1975 - Congress passes Public Law 93-638, the Indian Self- 
Determination and Education Assistance Act. “638” opens the door of 
opportunity for Indian tribes to contract programs from the federal 
government to conduct those programs for themselves. 

• 1976 - Indian Healthcare Improvement Act. Provided increased 
financial resources for services and research in the Indian Health 
Service. 

• 1978 - Indian Child Welfare Act - Allows tribes (in certain instances) 
to take complete control of Indian children’s custody, if the tribe 
chooses, and control the child’s placement in foster care and adoption. 

• 1978 - Indian Religious Freedom Act. Provides tribes and Indian 
people the right to practice all facets of their religion without 
persecution. 

• 1978 - Tribally controlled Community College Act - provides resources 
for the development of Community Colleges on reservations under the 
tribes’ control. 

• 1987 - Indian Tax Status Act - provided the authority for tribes to issue 
bonds to raise funds for tribal operations. 

• 1986 - Indian Alcohol and Substance Abuse Act - provides funds to the 
Indian Health Service to establish prevention and awareness programs in 
these areas. 



4 President Richard M. Nixon in his July 8, 1970, "Message from the President of the United 
States Transmitting Recommendations for Indian Policy,” House of Representatives Doc. No 91-363, 
91 s1 Congress. 
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• 1988 - National Indian Gaming Regulatory Act - provided tribes the 
opportunity to establish and operate gaming operations with a state 
compact. 

• 1988 - Public Law 93-638 Amendments - provides 10 tribes the 
opportunity to compact all their federal funds and allocate monies as the 
tribe chooses. 

• 1990 - Indian Law Enforcement Reform Act - created division of law 
enforcement in BIA and provided financial resources for these 
programs. 

• 1992 - Native American Graves Protection Repatriation Act - allows 
tribes to recover human remains and objects on federal lands; items on 
former tribal lands can be reclaimed by tribes. 

• 1992 - Indian Employment, Training, and Related Services 
Demonstration Act (Public Law 102-477) - provides tribes the 
opportunity to consolidate certain formula-funded grants into a single 
plan and seek waivers of selected program regulations relating to 
employment and training programs. 

• 1994 - Public Law 93-638 Amendments - provides 30 more tribes the 
opportunity to compact all their federal funds and allocate monies as the 
tribe chooses. 

• 1998 - Consistent with President Clinton’s Executive Memorandum, the 
Department of Labor establishes as its policy that all dealings with 
federally-recognized tribes are to be based on government-to- 
government relationships. 

As is evident, American Indian tribes do have a unique relationship in the United 
States. Their relationship with the federal government is a political relationship based 
on treaties and the recognition of tribes as sovereign independent nations. The U.S. 
Constitution reserves for the federal government the authority to make treaties with 
other nations and forbids the states from doing so, a restriction that many states have 
attempted to circumvent. For example, the states periodically attempt to assert state 
jurisdiction over tribes that reside in their state. Tribes resolutely oppose these actions 
and utilize the American system of justice (federal courts) to maintain their sovereign 
status. Similarly, when states attempt to levy state taxes on tribal enterprises (e.g., 
tobacco, etc.), the tribes fight these efforts, and lawsuits reach the Supreme Court 
annually; each time the tribes’ sovereign status has been upheld and the states’ petitions 
are rejected. 
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Thus, the tribes’ special relationship entitles them to concessions by the federal 
government that are based on treaty rights. Although each treaty has been broken, they 
are still in effect to this day. Only Congress can abrogate a treaty; the president cannot 
do so by executive order. Nor can the states impose their jurisdiction on tribes. As 
one consequence, Indian tribes have suffered under the federal government’s whims to 
make them “productive citizens.” Regardless, they have survived and even prospered. 
The difficult task for all entities involved is how to improve the quality of life of the 
Native American as a U.S. citizen and a citizen of their tribe. 

Conceptual Framework for the Evaluation 

Our technical approach to the evaluation has been guided by two heuristic 
models. The first is a client-level model, describing the elements of responsive 
services to INA program participants and the outcomes to which they give rise; the 
second is a system-level model, depicting the factors that promote or can constrain 
program quality. These models have guided our exploration of a series of descriptive, 
explanatory, and evaluative questions about the INA program. 

Client-Level Model of Responsive Services 

The client-level model of responsive services, presented in Exhibit 1-1, is firmly 
grounded in a conceptualization of high-quality services that draws on extensive 
research literature. This model depicts how clients flow through the program, the 
indicators of responsiveness for each phase of the service process, and the intended 
client-level and community- level outcomes. The model also indicates that the criteria 
for responsiveness hinge on the individual participant’s needs and interests, as well as 
on the goals grantees have established for their own programs in light of their 
community’s unique needs and circumstances. 

Outreach, Recruitment, and Assessment. As indicated in the exhibit, the first 
step is the effective outreach and recruitment of members of the Indian and Native 
American community. This includes efforts to ensure that all eligible members of the 
community have equitable access to services. Once recruited, an assessment of 
participants’ skills, needs, and interests should precede service planning. The 
comprehensiveness of the assessment is then tailored to what the program and the 
community can offer the participants in terms of employment and training, as well as to 
the participant’s motivations for entering the program. For example, some participants 
enter the Section 401 program with objectives that are clearly defined in their own 
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Exhibit 1-1 

Client-Level Model of Responsive Services 
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mind and sometimes very narrow in scope. Extensive testing in such circumstances 
could be viewed as unnecessary and even burdensome. Thus, assessment needs to be 
appropriate to the circumstances. 

Service Planning. The next critical step is to assist the participants in developing 
an individualized service plan that is tailored to their goals and interests. The role of 
Indian and Native American programs in service planning includes providing 
participants with information about the labor market in their immediate communities 
and in the broader labor market and providing employment and career counseling. 
Service planning should be a process in which the participant is actively involved, that 
helps the participant set short-term and longer-term goals, and that is attuned to cultural 
values and attitudes. 

Also as part of service planning, INA programs should be advocates for 
participants in helping them address skills deficits and in linking them up with needed 
support services. On the program level, the advocacy includes forming partnerships 
with service providers, brokering for needed services, and coordinating services to 
avoid duplication while ensuring the most comprehensive program of services possible. 
On the individual level, the advocacy should be client-centered and holistic in its 
approach. To the extent possible, case advocates need to work with each individual to 
address all needs to meet the participant’s goals for program participation. 

As shown in the exhibit, the next major step is to provide participants, either 
directly or through a network of providers, with the services called for in their 
individual service plans. These services include high-quality and responsive classroom 
training and worksite activities, job development/job placement, and support services. 
Participants access these services either singly, sequentially, or concurrently. As part 
of our conceptual framework, we have operationalized principles of high quality and 
responsiveness in each of these service areas. 

Classroom Training and Worksite Activities. The purpose of classroom and 
worksite training is to increase participants’ knowledge and skills in different areas of 
need, including pre- vocational as well as basic education and vocational training. 
Regardless of the type of training, evidence of high quality should include the 
specification of clear training objectives. These objectives need to communicate to 
participants what is required to be successful in the classroom or on the job and what 





skills are to be gained after completion. In both classroom training and work-based 
activities, well-specified training objectives facilitate learning and acquiring skills. 

A second criterion of high-quality training is that it promotes work-relevant skills 
in a functional context. Extensive research by cognitive psychologists demonstrates 
that training in a functional context is likely to be both more effective and more 
motivating. By teaching skills in an appropriate context, participants can recognize 
more clearly the importance of the skills they are learning and come to understand that 
what is learned will transfer to a real-world setting. As the SCANS report emphasizes: 

We believe... that the most effective way of teaching skills is ‘in context.’ 

Placing learning objectives within real environments is better than insisting 

that students first learn in the abstract what they will then be expected to 

apply. (Secretary’s Commission on Achieving Necessary Skills, U.S. 

Department of Labor, 1991, p. 19) 

Third, high-quality training programs use instructional methods that encourage 
active learning. Many studies have demonstrated that learning in which participants 
actively practice a new skill, or think about new information, enables them to acquire 
the new skill or knowledge more readily than when they are just passively listening or 
responding in a rote fashion or simply watching a skill being performed. Instructional 
methods that promote active learning include one-on-one tutoring, role-playing, 
project-based learning, cooperative learning groups, or supervised practice with 
instructor feedback. Many of these high-quality instructional practices are highly 
congruent with the emphases in many Native American cultures on cooperation and 
working with others. 

Fourth, high-quality programs promote training for transfer and skills durability. 
In other words, they not only teach new skills but also teach participants how and when 
to apply those skills for solving problems. Providing training in a functional context is 
an excellent way to foster training for transfer. Other training methods that enhance 
participants’ ability to transfer their learned skills to new situations include: 

• Demonstrating the skill’s effectiveness so that participants realize its 
benefits. 

• Giving participants rules for when a skill should be used. 

• Giving participants practice in when to apply specific skills in diverse 
contexts. 
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• Training participants to evaluate for themselves whether they are using 
the skills correctly. 

• Continuing training beyond the point of initial mastery of the skill. 

Fifth, high quality services provide varied and sufficient opportunities to learn. 
This means that instruction should be of sufficient duration and intensity to teach skills 
and knowledge. It also requires that participants spend substantial “time on task,” 
learning and practicing skills rather than engaging in irrelevant activities or waiting for 
further instruction. Finally, opportunities to learn include documenting participants’ 
progress in acquiring skills. These assessments should, ideally, (a) be ongoing and 
integrated into the instruction, (b) performance-based, using observations of 
participants’ skills in producing actual work products, and (c) involve participants in 
evaluating their own work and reflecting on their progress. 

Sixth, instructors or worksite supervisors should be sensitive to participants’ 
needs, motivations, and aspirations, and they should act as “coaches” or mentors rather 
than "directors” of participants’ activities. 

In general, these principles apply regardless of whether the training occurs in a 
classroom or at a worksite. However, in some instances subsidized employment (i.e., 
work experience and community service employment) serves only limited training 
objectives and is used instead as a short-term or stop-gap assignment for those 
undergoing job search or who need to test out an occupational area. In these cases, the 
training objectives and content might understandably be less well developed. In other 
instances, these activities are used primarily to provide valuable community services. 

In any case, high-quality subsidized employment needs to provide meaningful work and 
learning opportunities and be designed to advance community interests. And 
assignments of long duration should exhibit all the criteria for quality training identified 
above. 

Job Development/Job Placement. Although full-time, meaningful employment 
is one goal of employment and training programs, it may not be a realistic goal in some 
INA programs, particularly those in remote rural communities where full-time 
employment opportunities are virtually non-existent. Nevertheless, to the extent 
feasible, high-quality, responsive programs will try to assist their participants in 
obtaining appropriate and meaningful employment, either after training has been 
completed or as a direct placement (i.e., where participants do not intend to undertake 
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training). Indicators of high-quality and responsive job development and job placement 
services include those that: 

• To the extent feasible allow participants to successfully achieve their 
work and career objectives. 

• Develop jobs that use skills in which participants are being trained, or 
in the case of direct placements, make use of pre-existing skills. 

• Link placement efforts with the identification of potential new jobs in 
the local community or, if the participant desires, in the broader labor 
market. 

Furthermore, as part of the job development process, responsive INA programs 
seek to build the economies of communities in which they operate, to provide better 
opportunities for their people as well as to grow the communities themselves. 

Support for Training. Responsive programs also include support services that 
address the needs of participants as they engage in classroom training, work-based 
activities, or job search. Services may include help with purchasing tools or equipment 
needed for the job, family care, health care, personal counseling, nutritional assistance, 
or transportation assistance. As mentioned above, a holistic approach to case advocacy 
will help to identify the support service needs of participants that must be addressed if 
successful outcomes are to be expected. 

Outcomes. Finally, the far right box in Exhibit 1-1 identifies outcomes that high- 
quality and responsive programs are intended to achieve. The client- level outcomes 
include: (a) an increased awareness of employment and training opportunities; (b) 
enhanced skills that will further long-term employability; (c) increased earnings; (d) 
where feasible, employment that is consistent with the participant’s goals and is related 
to the training and/or services he or she received; (e) pursuit of longer-term goals 
through employment or further training; and (f) an increased self-esteem and cultural 
identity and sense of belonging to the community. 

Also important as a goal of many Indian and Native American programs is 
promoting community economic and social development. Community outcomes 
include: (a) better coordination of INA activities with other programs; (b) improved 
local services resulting from community service employment or work experience 
activities; (c) capacity building for agencies participating in the INA program; (d) 
leadership development among individual program staff and participants; and (e) a 
stronger sense of community culture and values. 



System-Level Model of Factors Influencing Responsiveness 

The system-level model, which is presented in Exhibit 1-2, illustrates how client- 
level services are shaped by the design and operation of the program and how they are 
influenced in turn by the broader context. The model includes both those actions 
within the control of the program (at the federal and grantee levels) and those factors 
outside program control (e.g. local economic conditions, geography, level of needs 
within the Indian or Native American community) . 

This system-level model has guided the evaluation of factors that influence the 
quality and responsiveness of IN A program services provided to participants. The 
exhibit illustrates aspects of federal policies and the local environment that can affect an 
IN A program’s design, and it shows how design decisions and provider characteristics, 
in turn, affect the quality of services provided. 

As shown on the left side of the exhibit, Federal policies and practices are 
important influences on the services offered through Indian and Native American 
programs. It is critical to trace through the influence of federal policies to determine 
whether existing policies have had their intended effects and to identify policies that 
DOL could adopt to improve services. These factors include: the funding levels 
provided for the DOL’s Title IV INA program; federal technical assistance and 
monitoring of grantees; federal program goals and regulations, including performance 
standards; the availability of funds to Indian and Native American communities from 
other federal programs, such as the U.S. Department of Education, BIA, HUD, and 
HHS; and special initiatives, such as the consolidation demonstration program 
administered by BIA (under Public Law 102-477). 

The local community context also is important in influencing program design and 
operations. For example, whether the INA program serves a community consisting of 
a tribe or multiple tribes or whether the program serves a reservation or a community 
that is not a reservation can have major impact on service needs and on the resources 
for providing services. Other local-context factors that are likely to influence how 
services are delivered and the resources available to provide different services include: 
the geographic and population size of the community to be served; local economic 
conditions; skills needed for jobs available in the community as well as in the larger 
labor market; characteristics of the local population interested in education, training, 
and/or employment; the availability of education and training providers; and the 
availability of other services desired and needed by the targeted population. 
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Exhibit 1-2 

Syslcni-Lcvel Model of Responsive Services for Indian and Nalivc American I’mgiam 
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In some of the IN A program communities, in particular many of the reservations, 
the dearth of job opportunities and the lack of appropriate service providers are major 
influences on the design and delivery of services for participants. The geographical or 
cultural isolation of some INA communities also exerts an influence on the design and 
delivery of responsive program services. In many cases, participants from these 
communities want to remain near their homes despite the lack of job opportunities in 
the area, and it remains the responsibility of the INA programs to develop as 
responsive a program of services as is possible under the circumstances. The lack of a 
well-developed infrastructure in many of these areas, such as a public transportation 
system that allows for movement within and to and from surrounding areas also largely 
impacts what an INA program can offer. 

Next, the central element in the system-level model is the INA program grantee 
itself. Influencing factors include grantee characteristics, such as the level of Section 
401 funding and the availability of other funding; staff experience and skills; program 
goals and objectives, including targeted participants and desired outcomes; service 
strategy; service delivery structure and provider arrangements; contract requirements 
and oversight of providers; and staff recruitment/training. Among these, a key factor 
will be the size of the grantee’s organization and the corresponding level of Section 401 
funding. Annual Section 401 funding to programs ranges from as little as $20,000 to 
as much as $7 million. Regardless of the needs of participants, such differences dictate 
varying degrees of ability to provide comprehensive services. The resources available 
to the programs, and their strategies for dealing with resource constraints (e.g., 
leveraging and coordination), will thus be important factors. 

Moving to the next box in the sequence shown in the exhibit, organizations 
providing education and training services also obviously affect the types and quality of 
services provided. Factors that can influence services include; service provider 
characteristics; instructor/supervisor experience and training; links between service 
planning, training/work experience, and placement; and links between training and case 
advocacy/tracking of client progress. The experience that service providers have with 
providing services to INA participants can affect the quality and responsiveness of the 
services. In many cases, providers will have other sources of funding, which may 
make it easier to provide more intensive services than INA program funds alone may 
allow, but may make the services less responsive to the goals and objectives of the INA 
program. 



The box to the far right of the exhibit includes the criteria for responsive services 
that we identified in the preceding exhibit. Criteria for high-quality and responsive 
services include services that are well-matched to the needs and desires of the 
participants; training, support, and placement services that are responsive to the needs 
and skills of participants; and services that result in desired participant and community 
outcomes. 
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II. DATA COLLECTION 



The research design for this evaluation consisted of two primary data collection 
strategies. The first was a mail survey administered to all Section 401 grantees. This 
survey served to capture key information from all grantees and to provide each of them 
an opportunity to provide meaningful input to the evaluation. 

The second and much more elaborate of the evaluation’s data collection 
components consisted of in-depth case studies that entailed site visits to a randomly 
selected sample of 23 INA grantees across the country. These visits each lasted 2-3 
days, and consisted of interviews with program administrators and staff, classroom 
instructors and worksite supervisors, and program participants and recent terminees. 
While on site, the field researchers also unobtrusively observed training services being 
provided and reviewed the case files of selected terminees to learn more about the 
service-planning process. 

In developing a strategy to use these data, the evaluation team was assisted 
throughout by a Technical Work Group, made up of members of the Indian and Native 
American Employment and Training Council, who volunteered to serve as advisors to 
the project. The 6 Work Group members gave generously of their time and were asked 
to provide input on all phases of the evaluation, including the design of the study, the 
data collection, and the analysis. Their assistance is greatly appreciated. 

The remainder of this chapter describes each of the project’s two major data 
collection efforts, beginning with the case studies, and then proceeding to the mail 
survey. 

Design of the Case Studies 

The several elements involved in conducting the case studies included: developing 
the sample, preparing for and conducting the site visits, and preparing write-ups. 

Developing the Grantee Sample 

Following the terms of our evaluation contract, we were slated to visit 23 
grantees as part of this study. In selecting these grantees, we needed to ensure that the 
final sample would be reflective of the grantee community as a whole as much as 
possible. This is no small challenge, because, as the preceding chapter has indicated, 
INA grantees and the communities in which they operate are extraordinarily diverse. 
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Although every program is in some sense unique, we needed to ensure that the grantees 
selected for the site visits reflected a broad range of variation on a number of key 
dimensions. For this reason, we developed a sampling plan that relied on stratified 
random sampling. 

Among the several relevant dimensions on which IN A grantees might be stratified 
for sampling purposes are these: 

• Program size. INA grantees include many with meager Section 401 
funding and others with allocations in the millions of dollars. Similarly, 
some programs serve handfuls of participants each year, while others 
serve over 1,000. Obviously this very uneven distribution of resources 
and scope gives rise to unique challenges for both small and large 
programs that the evaluation must investigate. For example, looking at 
the ways in which smaller programs leverage resources from other 
programs to stretch their job training dollars may provide insights of 
benefit to the entire JTPA community. 

• Grantee type. About two-thirds of INA programs are administered by 
tribal governments, and almost all of the remaining are run by 
community-based organizations. Implications of operating a program in 
the context of a tribal government are very different from those facing 
non-tribal entities. Thus, tribal government status was considered to be 
an essential dimension for sampling. 

• Community context. Related to some degree to grantee type, INA 
programs also vary greatly with respect to the contexts in which they 
operate. These span the range from sprawling, remote, and 
economically impoverished reservations, to small pueblo, to vibrant 
urban and tribal economies. 

• Regional variation. Each Indian tribe or other organization is to a 
large degree unique, and any generalization is bound to be noteworthy 
for the many exceptions to the rule. Nonetheless, some commonalties 
emerge that relate to regional variation in common histories, tribal 
cultures and lifestyles, and present-day environments. Typical 
groupings include the Eastern Woodlands, the Southeast, the Plains, the 
Great Basin, the Western Plateau, and the Pacific Northwest, among 
others. Similarly, grantees operating in Oklahoma are unique, because 
almost all operate as administrative arms of federally-recognized tribes, 
with a tribal government structure, but without land. 

Developing sampling strata that represent the intersection of each of these 
dimensions is obviously impractical— after all, the cross-tabulation of 4 dimensions 
with (say) three categories each yields 81 cells. After consultation with the Work 
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Group, our approach was to adopt a multi-step sampling procedure. First, we excluded 
from the sampling frame 9 grantees who were operating for the first time in program 
year (PY) 95, the year our study begin, on the grounds that these grantees would be 
experiencing the usual problems associated with start-up and, hence, would not be very 
representative of Section 401 grantees operating as mature programs. 

We also excluded grantees who received Section 401 funds but who were 
participating under Public Law 102-477. This act enables IN A grantees to consolidate 
their funds from various sources into a single service plan, and by waiving certain of 
the IN A program’s regulatory requirements, gives them substantial discretion to 
redesign their programs. Although the investigation of the efficiencies that can be 
realized by operating consolidated programs under streamlined regulatory requirements 
would make for a fascinating research endeavor, grantees operating under Public Law 
102-477 are not officially under DOL’s purview and, hence, these grantees could not 
be studied under this evaluation. At the time our sample was drawn, 12 grantees 
receiving Section 401 funds were operating under Public Law 102-477 and, hence, 
these were excluded from the sampling plan. 1 

After eliminating these two groups of grantees, the remaining grantees were 
allocated to one of five sampling strata. These were: 

• Oklahoma grantees. 

• Other (i.e., non Oklahoma) tribal-government programs, whose service 
area consists mostly of reservation territory; specifically included were 
those tribal programs with more than 50% of their service area 
population living on a reservation. 

• Other tribal-government programs, whose service area includes 
reservation but also substantial non-reservation territory; specifically 
included were those tribal programs with fewer than 50% of their 
service area population living on a reservation. 

• Non-tribal programs operating in large metropolitan areas; specifically 
included were non-tribal programs with more than 75% of their service 
area population living in metropolitan areas. 



1 However, other grantees, including several we did visit, were contemplating operating under 
Public Law 102-477, and a number have since decided to do so. 
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• Non-tribal programs operating in rural or small urban areas; specifically 
included were non-tribal programs with fewer than 75% of their service 
area population living in metropolitan areas. 

Table II- 1 shows the number of grantees in each of these strata at the time our sample 
was drawn, and the amount and percentage of PY 95 Section 401 funding that had been 
allocated to each. 



Table II I 

First-Step Sampling Strata 

# of PY 95 Allocations 



Category Grantees Amount % 



1. 


Oklahoma grantees 


20 


$6.3 


11.3 


2. 


Other tribal programs/high reservation 


51 


17.0 


30.2 


3. 


Other tribal programs/low reservation 


36 


9.9 


17.5 


4. 


Non-tribal programs/high metro 


31 


11.3 


19.9 


5. 


Non-tribal programs/lower metro 


28 


11.9 


21.0 



Note: The amount of allocations are shown in millions of dollars and represent the cumulative amount 
of PY 95 Section 401 funding received by grantees in that stratum, after excluding those participating 
in Public Law 102-477 or who were new grantees in PY 95. 



Of the 23 grantees to be included in the sample, we decided in conjunction with 
members of the Work Group that 21 should be drawn from these 5 strata. Selection 
was to be conducted randomly proportionate to size, both within and across the strata, 
with size measured by amounts of PY 95 Section 401 funding. Selection proportionate 
to size ensures that the resultant sample will be reflective of how the typical Section 
401 dollars are being spent. Given selection proportionate to size across strata, Table 
II-2 shows how many of the 21 grantees to be selected at this stage were drawn from 
each stratum. For example, since 11% of aggregate funds are represented by 
Oklahoma grantees, 11% of 21, or 2 grantees, should be selected from this stratum. 
Further, selection proportionate to size within strata ensures that larger grantees from 
within each stratum are more likely to be chosen than smaller ones. Thus, a program 
that is twice as large as another will have twice the probability of being selected. 
Finally, the selection algorithm was designed to increase the odds that the several 




II-4 



G L 60 



grantees selected from within each stratum would be drawn from different regions of 
the country. 

This general scheme should ensure adequate representation of very diverse INA 
programs, including tribal and non-tribal programs and those operating in very 
different community contexts. Moreover, selection proportionate to size ensures that 
the resultant sample will be reflective of how the typical JTPA dollars are spent. 
However, these procedures were also expected to yield very few instances of programs 
that are very small, which we will define here as programs with allocations of less than 



Table II-2 

Number of Grantees Selected from 
Each Stratum in the First Stage 

No. of Grantees 



Category Selected 

1 . Oklahoma grantees 2 

2. Other tribal programs/high reservation 6 

3. Other tribal programs/low reservation 4 

4. Non-tribal programs/high metro 4 

5. Non-tribal programs/lower metro 5 

Total 21 



$100,000. About 25% of the INA grantees in the sampling frame (i.e., after excluding 
new grantees or those participating in Public Law 102-477) are this small, yet their 
cumulative allocations amount to just $2.6 million, or less than 5% of the total. Thus, 
if all 23 grantees were chosen proportionate to size from the 5 strata defined above, 
only 1 grantee would be likely to be very small (or 5% of 23) . Understanding the 
challenges faced by small grantees was felt to be an important objective for the 
evaluation, especially since about one-quarter of all grantees are within this size 
classification. Thus, 2 case study sites were reserved for a second-stage selection. For 
this stage, 2 grantees were randomly selected from among all grantees in the sampling 
frame with funding of less than $100,000 and that had not been selected in the first 
stage. These two, when added to the 21 chosen in the first stage, yielded the 23 
grantees first selected for the study. 
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Each of these grantees was contacted by telephone by research staff and was 
urged to cooperate with the study. The Indian and Native American Employment and 
Training Council graciously agreed to add their considerable influence also; 
accordingly, Council members prepared a letter that was sent to each of the selected 
sites. Perhaps because of this gentle prodding, all programs that had been selected 
initially agreed to participate. However, as the site visits were being scheduled, one 
grantee who had been selected kept deferring the date for the visit. Consequently, a 
replacement grantee was randomly selected from within the same sampling stratum. 
These 23 constituted the final site visit sample. Figure II- 1 pinpoints their location on 
a map of the United States. 



Figure II-l 

The Location of Grantees in the Final Site Visit Sample 




Note: Symbols on the map represent the location of the central administrative office of each of the 
grantees selected for the site visit sample. 
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Characteristics of the Sampled Programs 

The sample design described above ensures that the sampled programs will reflect 
the range of grantee variation on key dimensions. As a way of summarizing the 
characteristics of the sample, however, Table II-3 presents the distribution of the site 
visit sample on key contextual attributes in comparison with all grantees taken as a 
whole. For the comparison, the distributions for all grantees are shown both 
unweighted and weighted by Section 401 allocations; in other words, the unweighted 
distribution counts each grantee equally, but the weighted distribution counts each 
grantee differentially according to the amount of its funding. Since we sampled 
grantees proportionate to size (with the exception of the two very small grantees 
selected in the second stage) , in general we would expect the distribution of grantees in 
the sample to match most closely with the distribution for all grantees after they had 
been weighted. 

These results show in fact that the distributions match quite closely on each of the 
dimensions shown. For example, about 60% of the grantees in our sample are tribal 
government programs and 40% are non-tribal programs, which matches quite closely 
with all grantees as a whole. Similarly, the regional distribution shows only minor 
discrepancies between the various columns. 

Differences are somewhat greater for the distribution on Section 401 allocations, 
but these differences are exactly as expected given the way we drew our sample. The 
first thing to note is the tremendous differences between the size distribution for all 
grantees when the sample is unweighted and when it is weighted. This comparison 
shows dramatically, as we suggested earlier, that IN A grantees are enormously diverse 
with respect to size. For example, about 25% of all grantees have allocations of less 
than $100,000 per year, but these make up only about 5% of all allocations. Similarly, 
at the other extreme, only 4% of the grantees have allocations of $1 million or more, 
but these make up 30% of total allocations. 

Because we drew 21 of the 23 grantees in our sample proportionate to their size, 
in general the distribution of allocations for our sample should match the weighted 
distribution of all grantees much more closely than the unweighted distribution, and 
indeed it does. Thus, about 40% of both our sample and the weighted grantee pool 
have funding from $100,000 to $499,000, about 20% have funding between $500,000 
to $999,000, and about 30% have at least $1 million in funding. The biggest 
discrepancy is in the smallest size classification, and this comes about by design. 
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Because we deliberately drew the final two grantees for our sample from the smallest 
size group (in the second-stage selection) , we in essence ensured that our sample would 
“split the difference” when compared with the weighted and unweighted distributions 
for all grantees. 



Table II-3 

Contextual Characteristics of Grantees 
in the Sample, Compared with All Grantees 



Grantees 
In Sample 



All Grantees 

Unweighte Weighted 
d 



Grantee Type 



Tribal 


60.6 


63.3 


57. 


Non-tribal 


39.1 


36.7 


42. 


Census Region 


Northeast 


4.3 


8.4 


5. 


South Atlantic 


8.7 


7.2 


5. 


East North Central 


8.7 


10.8 


7. 


West North Central 


13.0 


11.4 


10. 


South Central 


17.4 


17.5 


17. 


Mountain 


26.1 


28.3 


33. 


Pacific 


21.7 


16.3 


20. 


Section 401 Allocations 


Less than $100,000 


13.0 


24.7 


4. 


$100K to $499K 


39.1 


57.8 


40. 


$500k to $999 million 


17.4 


13.3 


25. 


$1 million or more 


30.4 


4.2 


29. 



Note: Figures represent percentages of grantees in the sample (in the first column) and in 
the sampling frame as a whole (in the second and third columns). The weighted percentages 
count each grantee in the sampling frame differentially, depending on the amount of its 
Section 401 allocation for PY 95. The sampling frame includes all grantees receiving 
Section 401 funding (at the time our study commenced), excluding those participating in 
Public Law 102-477 and a number of grantees who had just been newly designated. See text 
for further details. 
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We drew on the Annual Status Reports for PY 95 to make additional comparisons 
between our sample and the larger pool of grantees from which the sample was drawn. 
These results are shown in Table II-4. As the table shows, the outcomes recorded by 
grantees in our sample are very close to those posted by all grantees. For example, 
about one-half of the terminees from each group of grantees enter employment, an 
equal number attain an employability enhancement, and about 80% achieve a positive 
termination. 

The characteristics of terminees at intake are also very similar across the groups. 
Across all three groups, females terminees modestly outnumber males, and the age and 
education distributions are nearly identical. Finally, terminees in each of the groups 
record barriers to employment in about equal numbers, with appreciable proportions 
who are single heads of households, have multiple barriers to employment, are long- 
term unemployed, or receive public assistance. 

Thus, based on these tabulations we can conclude that the sampling strategy 
appears to have been effective in yielding a site-visit sample that reflects the range of 
variation of the grantee universe as a whole, at least on these key dimension. 

Conducting the Site Visits 

For each grantee selected for the sample, we conducted intensive on-site visits 
that lasted 2 or 3 days each, depending on the complexity of the site, as suggested by 
its amount of Section 401 funding. During each visit, we conducted in-depth 
conversations with grantee staff and community leaders. At each site, we also selected 
approximately 2 instances of service activities for detailed study and observation. 

These activities were selected purposively from among the various services provided by 
grantees to ensure some variation in activity type, both between and within sites. 

When pooled across grantees, the number of different instances of each service activity 
that were selected in this way included: 

• Basic skills classroom training: 3 instances 

• Occupational skills classroom training: 18 instances 

• On-the-job training: 6 instances 

• Work experience: 13 instances 

• Community service employment: 4 instances 
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Training assistance: 4 instances 






Table II-4 

Characteristics and Outcomes of Terminees Served by 
Grantees in the Sample, Compared with All Grantees 





Grantees 


All Grantees 




In Sample 


Unweighte 

d 


Weighted 


Outcomes 


Entered Employment Rate 


51.9 


51.9 


49.3 


Employability Enhancement 


50.9 


55.0 


53.6 


Rate 


Positive Termination Rate 
Characteristics of Terminees (%) 


81.9 


82.1 


81.8 


Gender 


Male 


41.3 


43.4 


43.2 


Female 


58.7 


56.6 


56.8 


Age 


Less than 22 


26.3 


29.6 


28.4 


22 to 29 


30.3 


30.3 


30.4 


30 to 54 


41.2 


37.8 


39.1 


55 or more 


2.2 


2.3 


2.1 


Education 


Less than high school 


20.3 


22.0 


21.1 


Student 


5.2 


8.0 


8.5 


High school graduate 


48.6 


47.6 


45.5 


Some college or more 


25.9 


22.4 


24.9 


Other Barriers 


Single head of household 


27.4 


27.4 


26.0 


Has multiple barriers 


38.1 


40.3 


36.4 


Unemployed long term 


39.6 


44.6 


38.1 


Public assistance recipient 


36.8 


34.3 


34.6 



Note: Data are drawn from the PY 95 Annual Status Reports submitted by each of the 
grantees and represent grantee averages (i.e., averages of the values reported by each 
grantee). The columns are defined as specified in Table II-3. 
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For each activity selected, we observed the service being delivered and spoke with the 
trainers (e.g., classroom instructors, worksite supervisors, etc.) and participants who 
were involved. 



Specific activities conducted in conjunction with each site visit included: 

• Reviews of existing information about the program, such as program 
plans and budgets, examples of RFPs and service contracts used with 
service providers, and summary reports on participant and/or terminee 
characteristics, services provided, and outcomes achieved by 
participants. 

• Detailed discussions with grantee directors and key administrative staff 
to understand local program goals and objectives and how they were 
established, how local staff developed a service strategy and designed 
services to meet those goals, and how service strategies and program 
operational structures address the needs and values of the Indian and 
Native American participants and communities. 

• Detailed discussions with local community representatives, including 
tribal officials or community leaders or representatives of other Indian- 
identified community agencies or organizations, to collect more 
information about local community needs, how INA programs have 
addressed those needs, and how the community has benefited from INA 
program activities. 

• Detailed discussions with grantee staff responsible for the delivery of 
upfront services, including recruitment, service planning, and case 
management, to understand service choices available to participants and 
the ability of those services to meet participants’ varied needs. 

• The in-depth study at each sample grantee of two representative service 
activities. For the activities selected for detailed study, we conducted: 

- Detailed discussions with provider staff, such as curriculum 
planners and classroom instructors. 

- Observations of services. 

- Reviews of written curriculum materials. 

- In-person discussions with participants in these services who 
were present at the time the activity was being observed. 

The purposes of these data collection activities were to describe the 
objectives for the particular service activity, document the 
service/training content and instructional or other service methods used 
to achieve service goals, and assess the quality of services and extent 
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that services were responsive to participants’ needs and values and 
furthered the employment/training objectives established by the grantee. 

• The review of existing information in client case files for approximately 
two to four recent terminees, to assess the nature of assessment, service 
planning, and case management services received by the individual . 2 

• Individual or small group discussions, conducted either in-person or by 
telephone, with these terminees to learn of their reactions to and 
satisfaction with the services they received. 

Before conducting the visits, however, we first developed field protocols to guide 
the on-site data collection. Separate guides were developed for interviews with 
different respondents and for other on-site data collection activities. Draft data 
collection protocols were shared with DOL and the Technical Work Group and were 
refined based on comments we received from them. The development of the guides as 
a formalized planning step helped to ensure that all project staff agreed about the scope 
and content of the case study visits. The guides were also to be used as a checklist to 
ensure that all relevant issues were covered during each site visit. Of course, these 
guides structured interviews only very loosely; site visit staff were encouraged to 
develop free-flowing discussions in their interviews and to build on each respondent’s 
special expertise and interests. 

Following the completion of each site visit, the field researcher prepared a 
narrative site visit report that described the results from the site visit activities. These 
narratives, which were viewed only by SPR and AIRD staff, had three main objectives: 
descriptive, explanatory, and evaluative. The first objective was to provide basic 
descriptions of program-level and service-delivery-level objectives, organizational 
structure, service strategy, and service delivery decisions and procedures, as well as 
information about the resource and geographic scale of each program and features of 
the local context that influenced or constrained the choices made by grantees and their 
service providers. 

The second objective was to understand the “how” and “why”— the factors that 
shaped the organization and service approach observed at each site and that affected the 



2 In so far as possible, we tried to ensure that these case files were randomly selected (e.g., by 
randomly picking names from rosters of recent terminees, or by pulling a case folder from a filing 
cabinet). In some instances, however, grantees had picked case files for us in an effort to be helpful. 
Although in even these cases we instructed the grantee staff to select randomly, we had little control 
over the actual selection. 
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grantee’s or provider’s ability to provide responsive services. This second level of 
understanding was essential for identifying intervention points (parts of the system 
where changes in policy or management might support the improvement of the INA 
program as a whole or in specific sites) , as well as for understanding how strategies for 
program improvement may differ for grantees facing different external environments or 
with different internal organizational structures. 

The third objective was to assess the extent to which the overall program and 
individual service approach and program operations met the criteria for responsive and 
high quality services described in the conceptual framework and to identify the 
strengths and weaknesses and opportunities for program improvement in each local case 
example. 

This Final Report was developed by drawing heavily on these write-ups. 

The Mail Survey 

An additional data-collection component was an Administrators Survey, which 
was to be completed by all Section 401 grantees (except those participating in Public 
Law 102-477). This survey, along with data drawn from the Annual Status Reports, 
was intended to provide a baseline of information for all programs that would round out 
the in-depth site visit information we collected from the sample. The survey was 
additionally viewed as an opportunity for all grantees to provide input into the 
evaluation, by allowing them to draw attention to issues that concerned them. 

The survey was designed with these specific objectives in mind: 

• To assess how satisfied the grantees are with the services and guidance 
received from the Department of Labor, including those relating to 
training and technical assistance, performance standards, and reporting. 

• To understand program goals and emphases that recipients of Section 
401 funds have established for themselves, in relation to organizational 
structure and context. 

• To determine the greatest obstacles the grantees face in obtaining 
successful outcomes for participants. 

An initial draft of the survey was prepared in the fall of 1996 and was submitted 
to DOL for its review. Shortly thereafter, copies were distributed to members of the 
Native American Employment and Training Advisory Council during a meeting of the 
Council held in October of 1996. Later, nearly 100 copies were distributed to 
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members of the grantee community at a regional conference of IN A grantees that was 
held in November 1996 in South Dakota. Finally, members of the Work Group 
specially designated by the Council to advise the evaluation team reviewed multiple 
drafts of the survey, beginning in late 1996 and extending into the first few months of 
1997. 



In all these cases, reviewers were asked to comment on any aspect of the 
questionnaire, including its clarity and content. In an iterative process, comments from 
initial rounds of reviews were used to modify the survey before sending it out for 
additional feedback from others. By the end of March 1997, after the survey had 
undergone several revisions, a final draft was ready for review by the federal 
government’s Office of Management and the Budget (OMB) , whose go-ahead was 
necessary before the data collection could proceed. OMB had its own comments, and 
the survey was revised yet another time, before approval was finally given in December 
of 1997. A copy of the survey is included as Appendix A. 

The mail-out then commenced in early January 1998. Each grantee receiving 
Section 401 funds, except those participating in Public Law 102-477, was mailed a 
copy of the survey, along with a letter from Mr. Thomas Dowd, Chief of DINAP, 
encouraging grantees to respond, and a postage-paid business reply envelope. 

Following our standard procedures for surveys of this sort, we mailed reminder 
postcards to all non-responding grantees approximately three weeks after the initial 
mailing, and then mailed out another copy of the questionnaire approximately three 
weeks after that to those who still hadn’t responded. Additionally, several advisory 
Work Group members publicized the survey in various regional meetings, and Mr. 
Norm DeWeaver, a Work Group member, gave the survey a plug in his widely 
disseminated Friday Report. In an effort to get still more returns, AIRD and SPR staff 
telephoned each of the remaining non- respondents to encourage them to reply. These 
calls generated another batch of returns, before the processing of surveys was finally 
cut-off, in late May. Table II-5 shows that, in total, 113 of the 165 grantees returned 
completed surveys, for a very satisfactory final response rate of nearly 70%. 

Although a 70% response rate is quite impressive, we still must be concerned that 
non-respondents would have given different answers than those for whom we do have 
data. To the extent this is true, our survey findings might not generalize very well to 
the entire grantee community. The existence of such potential “non-response 



Table II-5 

Response Rate to the Administrator Survey 

Number Percent 

Returned a completed survey 113 68.5% 

Did not return a survey 52 31.5 

TOTAL 165 100.0 

Note: Figures represent the number and percentage of grantees. 



bias” can never be refuted conclusively, because by definition we cannot know how 
non-respondents would have answered the survey. We can, however, test whether non- 
respondents are systematically different from respondents on other measured 
characteristics, such as their tribal status, the size of their funding allocation, the 
outcomes they record for participants, and so on. If there were such differences, we 
would at least know what segments of the grantee community were underrepresented by 
our data; on the other hand, if differences were modest, we would have more 
confidence that the survey results in fact reflect the sentiments of the grantee 
community at large. 

Table II-6 presents these differences, showing the sampling strata, tribal 
government status, Census region, funding amounts, and the average outcomes attained 
by participants, for grantees that did and did not return an Administrator Survey. As 
the table shows, almost all differences are quite small. In fact, only one difference 
attains statistical significance at the .10 level. Thus, with this one exception, chance 
cannot be ruled out as an explanation for the modest difference in characteristics 
between respondents and non-respondents. For example, tribal programs were about as 
likely to complete the survey than non-tribals, as were those in various regions of the 
country, and with different funding amounts. Even outcomes, as reflected in the 
entered employment rate and employability enhancement rate, are approximately equal 
between responding and non-responding grantees. The one difference that is significant 
suggests that non-tribal programs in heavily urban areas were somewhat less likely to 
complete the survey than were grantees in other strata. 

Overall, then, we conclude that no appreciable evidence of response bias is 
uncovered in these data. 
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Table II-6 

Characteristics of Grantees, by 
Whether or Not They Completed the Administrator Survey 





Returned a 


Did not Return 




Survey 


a Survey 


Primary Sampling Strata 


Oklahoma grantees 


12.1% 


10.2% 


Other tribal/high reservation 


32.7 


26.5 


Other tribal/low reservation 


21.5 


20.4 


Non-tribal/high metro 


15.0* 


26.5 


Non-tribal/lower metro 


18.7 


16.3 


Grantee Type 


Tribal 


65.1 


58.8 


Non-tribal 


34.9 


41.2 


Census Region 


Northeast 


7.3 


9.8 


East North Central 


11.0 


7.8 


West North Central 


12.8 


9.8 


South Atlantic 


8.3 


5.9 


South Central 


18.3 


13.7 


Mountain 


27.5 


31.4 


Pacific 


14.7 


21.6 


Section 401 Allocations 


Less than $100,000 


24.8 


27.5 


$100K to $249K 


35.8 


25.5 


S250K to $499K 


25.7 


21.6 


$500k to $999 million 


11.0 


17.6 


$1 million or more 


2.8 


7.8 


Outcomes 


Entered Employment Rate 


54.9 


54.5 


Enhancement Rate 


57.4 


54.6 



Note: Figures represent the percentage of grantees with the characteristics in question, 
except for rows tabulated under Outcomes, which represent the mean entered 
employment rate and enhancement rate for grantees in the two columns. Significance 
tests are computed as a comparison of mean differences across columns. 

* Statistically significant at the .10 level. 

** Statistically significant at the .05 level. 
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III. RESULTS FROM THE ADMINISTRATOR SURVEY 



Although our site visits provided us with the opportunity to learn in-depth about 
the service designs of a small number of programs, the Administrator Survey enables us 
to look more broadly. As noted in the previous chapter, about 70% of all grantees 
completed the survey, giving us good confidence that the data generated from this 
source reflect the sentiments of all segments of the grantee community. 

In addition to answering the closed-ended questions, grantees who responded to 
the survey also took the time to provide extensive written comments on a range of 
topics, including their program goals, their needs for technical assistance, suggestions 
for improving performance standards, their thoughts on barriers to serving their clients, 
and so on. In fact, scarcely a dozen of the 113 grantees who returned surveys failed to 
provide written comments of some sort, and most wrote extensively on multiple topics. 
We are extremely gratified and grateful that so many grantees took the time to share 
their thoughts with us. The responses to the close-ended survey questions, as well as 
these extensive written comments, thus provide a rich source of information about the 
sentiments of a broad spectrum of the grantee community on a range of topics. 

In this chapter, we summarize the survey responses and will quote extensively 
from the comments the grantees provided, so that grantees can express their thoughts in 
their own words. 1 We emphasize, though, that the verbatim grantee comments rightly 
reflect only the opinions of the grantees who made them and should not be construed as 
representing the opinions of the grantee community as a whole. 

Relationships with the Department of Labor 

As part of the Administrator Survey, we asked the grantees an extensive battery 
of questions about their interactions with DOL, including whether policy guidance was 
clearly expressed, whether their technical assistance needs were being met, whether 
reporting requirements were appropriate, whether the consultation process was 
adequate, whether performance standards captured important dimensions of their 
performance, and so on. In each of these areas, grantees were asked to respond, 



1 The comments from the grantees have been slightly edited, to improve clarity (e.g., by 
eliminating abbreviations, adding articles and connecting words, etc.). 
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typically on a 4-point scale, whether they agreed or disagreed with each of various 
statements; those reluctant to express an opinion had the option of circling “no opinion” 
as their response. 2 Grantees also had the opportunity to provide written clarification of 
their response or give additional opinions or express concerns. These findings will be 
described in this section. 

Policy Guidance 

Among the most important functions of DOL’s Division of Indian and Native 
American Programs (DINAP) is providing an overall direction to the Section 401 
program. In carrying out this responsibility, DINAP, in cooperation with its grantee 
partners, crafts regulations, issues other guidance, and by its actions in general sets an 
overall tone that establishes program priorities. 

As part of the Administrator Survey, we asked how clearly this program guidance 
was being communicated and whether it provided sufficient flexibility for grantees to 
address the needs of their participants. These results, shown in Table HI-1, suggest 
that grantees overwhelmingly believe that policies are being communicated clearly, 
with just short of half endorsing this sentiment strongly and nearly another half 
endorsing it somewhat (question ‘a’ in the Table). 

Grantees also feel that policies give them the flexibility they need to get things 
done. For example, fewer than 10% disagree with the statement that “our program has 
enough flexibility to address the needs of participants” (question ‘b’), with over half 
agreeing with this statement somewhat and another 38% agreeing strongly. 

Regulations are, however, perceived as being somewhat too complex, at least by 
about 60% of the respondents (question ‘c,’ in Table III-l). Consequently, it comes as 
no surprise that efforts currently undertaken by DINAP, as part of the partnership 
initiative, to revise the regulations by simplifying them are viewed very favorably 
(question ‘d’). Indeed, no statement in the section receives a stronger endorsement, 
with over two-thirds agreeing strongly and nearly everyone else agreeing somewhat that 
the “partnership efforts to change the regulations are greatly needed.” Moreover, this 
initiative is also viewed as giving grantees an important voice in how new regulations 



2 Responses of “no opinion” are generally not shown in the tables to follow; percentages in the 
remaining categories were calculated after excluding responses of “no opinion” from the base. Except 
where noted, very few grantees gave “no opinion” as their response. 
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and other policies are crafted, with almost all grantees agreeing strongly or somewhat 
that it is now easier for grantees to have input on policy issues (question ‘e’). More 
generally, the partnership effort is viewed, in this section of the questionnaire and 
others, as a fresh wind, with grantees consistently giving this initiative their highest 
endorsement and citing it as instrumental in ushering in a new atmosphere of trust and 
co-responsibility. 

It appears, then, that program policies are communicated clearly to grantees and 
are not perceived as being overly stringent or unreasonable. Nonetheless, the program 
regulations are viewed as more complex than they need to be, so DINAP’s recent 
efforts as part of the partnership initiative to streamline them are viewed very 
favorably. 



Table III-l 

Opinions about Policy Guidance 








Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


a. DINAP clearly communicates the 
program’s policies 


47.7% 


45.0% 


7.2% 


0.0% 


b. Our program has enough flexibility to 
address the needs of participants 


37.5 


54.5 


6.3 


1.8 


c. Program regulations are too complex 


16.4 


43.6 


30.9 


9.1 


d. The recent partnership efforts to change 
the regulations are greatly needed 


67.6 


27.8 


0.9 


3.7 


e. DINAP’s partnership initiative makes it 
easier to have a say in program policies 


56.4 


36.4 


7.3 


0.0 


Note: Figures represent the percentage of grantees who gave the responses indicated, after removing 
from the base the small number of grantees who gave a response of “No Opinion.” 



Technical Assistance 

In partnership with the National Indian and Native American Employment and 
Training Conference, DINAP co-sponsors training and technical assistance designed to 
communicate policy guidance and assist grantees in improving their program 
management. DINAP also periodically issues Bulletins and maintains a Web site, and 
its staff are available to provide support over the telephone or in person. As part of the 
Administrator Survey, grantees were asked whether their technical assistance needs 
were being met through these methods and how technical assistance could be improved. 
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Table III-2 shows that overwhelmingly grantees value the types of technical 
assistance they are being provided. Over 90% agree somewhat or strongly with each of 
the statements to which they were asked to respond, signifying in each case satisfaction 
with the mechanisms being provided to meet their technical assistance needs. Despite 
this strong and consistent expression of satisfaction, grantees offered numerous 
suggestions for how technical assistance could be improved. In fact, we received more 
comments on technical assistance than on any other topic covered by the survey; all 
together, over 70 grantees wrote comments on this subject. 

For example, more than one-third of the grantees agree strongly and over one- 
half agree somewhat that the annual national conference “meets the needs of grantees” 
(question ‘a’ in the table). However, despite this solid support, grantees do see areas 
where improvements are desired. 3 For example, some feel that individual workshops 
could be improved: 

The national conferences are extremely helpful, (but) some presenters could be better. I 
would also like more hands-on training, and I want to hear more about what other grantees 
are doing. 

Provide information to support work-to-welfare. 

Add workshops to increase participants’ motivation and excitement. 

Bring in experts from other federal job training agencies to conferences as presenters, to 
provide a broader perspective. 

Some grantees also suggested that the workshops could be better tailored to meet 
the particular needs of subsets of grantees: 

Many of the workshops at the national or regional conferences are designed for the big non- 
reservation programs... Workshops should include ... ideas for small and reservation 
grantees, on how best to use our money. 

For technical assistance at the national and regional conferences, (it would be) beneficial to 
have beginner, intermediate, and advanced levels. 

It would be helpful to set up workshops at the national or regional conferences according to 
reservation/non-reservation status and according to the size of the grant. 

TA for experienced Directors ... should be in a round-table format or casual discussion 
group so that each person’s expertise is utilized. 



3 It should be noted that, although DOL supports the conferences financially and otherwise, they 
are generally planned and conducted by the grantees themselves. 
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Table III-2 

Opinions about Technical Assistance 





Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


a. The technical assistance at the national 
conference meets the needs of grantees 


38.9% 


52.8% 


5.6% 


2.8% 


b. The one-on-one Fed. Rep. /Grantee 
sessions held at conferences help meet 
specific needs 


65.1 


33.0 


0.9 


0.9 


c. The regional conferences provide a good 
oppty to get assistance from fellow 
grantees 


60.7 


34.6 


1.9 


2.8 


d. 1 support the use of peer-to-peer technical 
assistance 


73.6 


23.6 


1.8 


0.9 


e. The DINAP Bulletins provide important 
information about program requirements 


76.6 


21.6 


1.8 


0.0 


f. I access the Internet to obtain information 
from DINAP that benefits my program 


46.8 


43.0 


5.1 


5.1 


g. DINAP’s partnership efforts have 
increased my willingness to ask questions 
when needed 


57.0 


39.3 


2.8 


0.9 


h. DINAP staff provides accurate 
information when our program asks 
specific questions 


56.0 


39.4 


3.7 


0.9 


i. DINAP answers our questions promptly 


63.9 


28.7 


5.6 


1.9 



Note: Figures represent the percentage of grantees who gave the responses indicated, after removing 
those who gave a response of "No Opinion” from the base. Only small numbers of grantees gave an 
answer of "No Opinion,” except for item f, on which about one-quarter of the grantees gave this 
response. 



Sentiments on using federal dollars to defray the transportation expenses of 
smaller grantees were mixed. One grantee wrote that DOL should: 

Increase funding to accommodate 2-3 persons to attend the national or regional conferences 
for training and TAT. 

But another felt that: 

I do not believe that the most effective use of TA funds is transporting small grantees to 
national conferences... These grantees would be better served by regional TA and peer-to- 
peer TA. It is nice to have them at the national conferences, but I don’t feel the dollars 
spent are effective in providing for their specific needs. 




III-5 



77 



And, thinking of those who never make it to the conferences, one respondent 
suggested that: 

If the grantee is unable to attend the national conference, send them the program year 
booklet. 

Although the national conferences were much valued, some felt that regional 
meetings should also be promoted, to defray the time and expense involved in 
transportation: 

Need to break down regions into smaller regions and conduct workshops into each small 
region, so that grantee transportation will not be as far. Thus, more grantees will be able to 
participate. 

We would like regional meetings held in a location within our own state. Don’t like 
traveling long distances. 

As shown in Table III-2, another value of the conferences was the opportunity 
they afforded for the one-on-one meetings between grantees and DOL’s Federal 
Representatives (Fed Reps), which were almost unanimously viewed as “helping to met 
grantees’ specific needs” (question ‘b’). But, on this issue again, grantees expressed 
diverse sentiments and opinions. Some felt that: 

We need more than 15 minutes to meet with our Fed Rep one-on-one at the conferences. 

The Fed Rep/grantee one-on-one should include a “report card” of how the grantee is doing, 
so annual goals can be focused on program improvements. 

Others felt that individual sessions with Fed Reps were so valuable that grantees 
needed additional opportunities of this sort: 

Quarterly regional meetings with Fed Reps would help. Regional meetings provide a more 
informal ... atmosphere and more people tend to open up. 

There should be periodic regular contact with Fed Reps, at least by phone... There is not 
enough direct contact. 

I believe if there were time and money, it would be of great benefit to me to have a ... visit 
from a Fed Rep to see how our program would rate. 

(It would be nice to have) one hour per month with our Fed Rep. It wouldn’t be mandatory, 
but (it would be nice to know) you could reach your Fed Rep that day. 

Conferences have worked well for us... (but) nothing beats being able to meet Fed Reps one- 
on-one at other times of the year. Out-stationing, such as the efforts currently being 
undertaken, may be the solution. 

Outstationing was also viewed favorably by another grantee who wrote: 

We are finally receiving the assistance we need by having the Fed Rep move into our area. 
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More generally, numerous grantees wrote comments on the nature of their 
relationship with their Fed Reps. Overwhelmingly, Fed Reps are viewed in a very 
favorable light, with over 90% of respondents agreeing strongly or somewhat that Fed 
Reps provide accurate and timely information (questions ‘h’ and ‘i\ in Table III-2). 
Moreover, the recent partnership initiative, in another strong endorsement, is viewed as 
“increasing my willingness to ask questions from the Fed Reps when needed” (question 
‘g’, in Table III-2). Written comments reaffirmed these judgements and show that: 

The DINAP Fed Reps respond quickly and work hard to answer questions... I feel like 
DINAP staff act as advocates, which is a new role for them and very well received. 

In the last two years, technical assistance has improved so much. ..I don’t have any problems 
talking with my Fed Rep. 

The Fed Rep has proved very effective. 

I get the help I request. 

My Fed Reps help me a lot; they always find the answers to our questions. We have a very 
good relationship. 

But some grantees have not been so fortunate. Despite the strong positive 
responses from most respondents, others noted some difficulties in the help their Fed 
Reps provided, or offered suggestions for improvements: 

Answers provided by Fed Reps sometimes vary from one to another. (There is) too much 
specialization among Reps and sometimes the only person who could answer your question 
is not available. 

There is nothing more frustrating than talking to DINAP about an issue/problem than to be 
left in the lurch because the resolution or answer is not forthcoming. 

DINAP staff seem stuffy and unapproachable, making me feel inferior. 

Response time is not good. 

I asked about... and was unable to get a straight answer. 

More understanding from Fed Rep on all issues concerning JTPA programs: have had 
problems with Fed Rep assigned to our program in just basically understanding. 

It would be nice to get a clear yes or no, without the uncertainty that is often present. 

Fed Reps need to familiarize themselves with tribal structure and how things operate within 
tribes. 

To improve TA would be just to have Fed Reps understand what problems we face as a 
grantee located in a remote area. 

Thus, although Fed Reps are generally viewed in a positive light, there have apparently 
been some problems with individual interactions. 
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If face-to-face meetings with Fed Reps are highly desired, so are peer-to-peer 
exchanges. Indeed, many grantees express a strong preference for receiving technical 
assistance in this way. For example, the survey showed that over 70% of grantees 
strongly support the use of grantee-to-grantee technical assistance, and another 24% 
support it somewhat (question ‘d’ in Table III-2). And respondents wrote: 

Shift more emphasis to peer-to-peer. 

More emphasis should be placed on peer-to-peer consultation. Personally, I believe this 
method is more effective. 

Peer-to-peer TA provides more assistance, as grantees basically know what problems we are 
facing in remote areas. 

But peer-to-peer exchanges are not always easy to arrange, as some grantees 
noted: 

The opportunity is there, but with so many grantees from various backgrounds, it is not easy 
to get the assistance from fellow grantees unless we find one ... that is similar to us. 

Still, this strategy might work better if DOL would: 

Disseminate information about peer-to-peer technical assistance... such as how to request it, 
who can provide it, cost limitations, etc. 

With respect to disseminating information, DINAP Bulletins have been a long- 
standing means used by DOL to relate important program guidance. These Bulletins 
are overwhelmingly valued as an important source of information, with 77% of 
grantees agreeing strongly and another 22% agreeing somewhat that “DINAP Bulletins 
provide important information about program requirements” (question ‘e’, in the table). 
However, a few grantees noted that the Bulletins could be written more clearly, or that 
other written material would also be helpful: 

We would like to see an update to the Director’s Manual. The organization of the manual 
can sometimes make it difficult to locate necessary information. 

I would like a more defined WorkBook on each service we offer — what we can do and what 
is not allowed. A WorkBook where we could turn to a page and find what we need on CSE, 
WE, SS, etc. 

Examples of actual participant files would be of great help. 

The only thing is that some DINAP Bulletins are written in “Washingtonese” and are a bit 
hard to translate. 

The electronic exchange of information is another method of technical assistance 
increasingly used of late and one about which many grantees had strong opinions. In 
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the survey, about 90% of those giving an opinion agreed with the statement that “I 
access the Internet to obtain information from DINAP that benefits my program” 
(question ‘f’). Some respondents also had suggestions for how the use of the electronic 
highway could be expanded: 

Since most grantees have e-mail, I think it would be great if there was a listProc, where all 
of us could post our questions and receive answers from other grantees. 

Provide distance TA through Internet access, with on-line real-time interactions. 

(Make the) Friday Report available on line; it would be less expensive. 

Identify on the Web site what types of assistance are available. 

Allow grantees to fill out evaluations of conferences on line, after returning to our offices. 

Have all DINAP bulletins on line and use pdf format for larger documents. Utilize the 
Internet to send reports electronically. 

The Home Page and Netforum need to be kept more current. 

Use the Internet to provide discussion groups... Scheduled discussion times and sites should 
be established with prearranged topics; grantees wishing to participate could have advanced 
notice of time and topics to be discussed. This method ... would allow for either direct 
participation or merely observation for those who desire. 

Perhaps a question/answer from grantee to Fed Rep could be written on the chat line. Other 
grantees might have the same question and like very much to know what other Fed Reps 
might suggest. Also, grantee-to-grantee questions on the web (would be useful, as a way of 
seeing) how each handles specific problems. 

However, some are clearly less comfortable with electronic interactions than 
others. For example, the question on Internet access was the only one in this section of 
the questionnaire that generated an appreciable number of ‘no opinion’ responses. 
Moreover, some grantees wrote: 

We’re still having problems... We need help to get this to work (but) DOL staff are not 
always available to answer questions. 

Develop software that is user-friendly. 

Computer training at the national and regional conferences isn’t helpful, because we don’t 
know what the problem is with our computer. 

Another area in which grantees feel the need for more technical assistance is in 
the information available for new Directors. Considering that not many respondents 
were new Directors, the number of comments we got on this topic was rather startling 
and suggests that new Directors need a better orientation to JTPA than they have been 
receiving: 
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I feel that the technical assistance I sought at the National Conferences was unmet. It would 
be most helpful to have a thorough introduction to the JTPA program for new Directors. 

The previous Director left and no one is here to train the new Director. I’m learning by 
trial and error. 

More training is needed for new Directors... Hands on and show me type of workshops. 

Need more TA for new Directors... I more or less had to fend for myself because I didn’t 
know what was offered ...New programs should be given TA immediately. 

When there is a new Director, it would be helpful to have an orientation on all aspects of the 
program. 

Finally, we offer an assortment of other comments relating to various technical 
assistance and training issues: 

(Provide) more funding alerts. 

DINAP’s response is good, but other DOL areas (e.g., accounting, OMB) are not in their 
control. Sometimes what DINAP staff says directly contradicts what these other offices say. 

TA should be provided without being cited for a deficiency. 

(We need) TA for operating in a rural area with very little funds. 

Reservation programs receive more technical assistance than urban programs. Provide more 
technical assistance that is relevant to urban programs. 

(We want) more on-site TA. 

During the past four years, DINAP has made every effort possible to provide good technical 
assistance. However, if grantees are not aware of problems, they cannot request assistance. 
Many grantees may not be aware of difficulties ... until it is too late. 

We want training seminars just for Directors, separate from the regular conferences. 

A well-run program with experienced staff shouldn’t need very much help from DINAP. 

Consultation 

The partnership efforts undertaken by DINAP, grantees, and the Native American 
Employment and Training Advisory Council have attempted to improve communication 
and dialogue and give grantees an explicit role in decision-making, reversing the 
adversarial atmosphere that had permeated the grantee-DINAP dialogue in years past. 
We asked grantees about the consultation process, and whether they thought the 
partnership initiative had indeed made them feel that their voices could be heard. 

In their responses, grantees express overwhelming support for the recent changes 
to the consultation process. For example, Table III -3 shows that two-thirds of grantees 
strongly agree and almost everyone else agrees somewhat that the partnership effort has 
made working with DINAP a more positive experience, and nearly as many agree that 
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it has helped establish mutual trust and respect (questions ‘a’ and ‘b\ of Table III-3). 
Grantees also generally believe that DINAP does a good job of getting opinions from 
grantees before making decisions, with over 40% agreeing strongly and another 44% 
agreeing somewhat (question ‘c’). As several grantees noted: 

I feel DINAP is working hard to support grantees and improve communication. 

The longer the partnership continues, the stronger will be the trust bond and the better the 
overall programming. 

Mr. Dowd has truly made every effort possible to include the grantee community in 
decision-making, problem-solving, and program changes... I believe we truly are partners 
with DINAP for the first time. 

However, some grantees, even if only a minority, feel that not everyone’s voice 
is being heard, either by DINAP or the Advisory Council: 

Partnership positions should rotate so all can be actively involved. 

Constantly work with grantees, and bring new grantees into the workgroups, so that work 
and responsibility can be shared. 

DINAP uses established committees as a means of consultation... However ... rarely are 
reservation program Directors included on these committees. 

We need to make sure all grantees are represented. Sometimes DINAP doesn’t consult 
enough with reservations or on Tribal Government issues. 

Advisory Council members should keep in better touch with local grantees; they represent 
all of us. 

We need healthy dialogue at regional and national conferences ... Include other views 
besides the Advisory Council and workgroups. 

Reinforcing support for electronic communications that was described above with 
respect to technical assistance, respondents overwhelmingly agree that “the Internet is a 
good way of giving and receiving information” for purposes of consultation (question 
‘d’, Table III-3). However, they had some suggestions for how this medium could be 
used more effectively: 

Develop a newsgroup or listserv (for electronic consultation) . 

(We recommend) more effective use of the Internet, particularly on the part of the Advisory 
Council. 

(We want) regular updates from DINAP once a month, relayed through the Internet. 

Grantees also had other suggestions for how consultation could be improved: 

Important areas are outside of DINAP’s control. Could they be brought into the 
consultation process? 
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Guarantee funding; grantees are non-confrontational out of concern for future funding. 

Fed Reps should pay "consultation” visits, rather than merely site monitoring. Consultation 
visits would fall more into the partnership attitude... and would be more conducive to 
communication and exchange. 

Grantees working one-on-one with DINAP have the trust and respect for them. Grantees 
that do not have continual contact are still leery. 



Table III-3 

Opinions about the Consultation Process 





Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


a. The partnership effort has made working 
with DINAP a more positive experience 


67.0% 


29.4% 


1.8% 


1.8% 


b. The partnership effort has established trust 
and respect between DINAP and grantees 


58.5 


34.0 


6.6 


0.9 


c. DINAP does a good job of getting 
opinions from grantees before making 
decisions 


41.7 


44.4 


12.0 


1.9 


d. The Internet is a good way of giving and 


65.6 


28.0 


6.5 


0.0 



receiving information 



Note: Figures represent the percentage of grantees who gave the responses indicated, after removing 
those who gave a response of "No Opinion” from the base. Only small numbers of grantees gave an 
answer of “No Opinion,” except for item d, on which 16% of the grantees gave this response. 



Reporting 

As a condition of their grants, grantees are required to submit periodic reports. 
As part of the partnership initiative, DOL has worked with grantees over the last 
several years to ease the reporting burden, by streamlining reports and decreasing the 
frequency of reporting. We asked a series of questions in the survey about this 
initiative. Grantees also were asked to write their comments, and many did so. 

In general, the recent revisions to the reporting forms are well received. As 
Table III-4 shows, almost all grantees agree strongly or somewhat with the statement 
that recent revisions have made the forms easier to use (question ‘a’). As some noted 
in their comments: 

DINAP deserved kudos in this regard... Streamlining the overall process is going well. 

The reporting forms are excellent. 




Ill- 12 



84 



Nonetheless, an appreciable minority of the grantees (just over one-third) still find 
the forms too complicated (question ‘b’). 

Recent changes are a big improvement, but some line items are still confusing and we need 
to keep referring back to the reporting instructions. 

Terminology is a bit much; too much jargon. 

Definitions can be interpreted differently than intended. Definitions need to be more clear 
and specific. 

Perhaps simpler explanations rather than lengthy wordy sentences. 

(We want) simple instructions and less time filling complicated forms (to save) more time 
for clients. 

We also asked grantees if they felt that the Annual Status Report covers 
everything important about their programs. Almost 90% of respondents believed that it 
did, even though only about 40% thought so strongly. A number of grantees also gave 
suggestions for what they thought should be added: 

Many positive outcomes are not reportable on the form.... Grantees should be able to report 
outcomes that are uniquely relevant to them. 

Time and assistance and coaching in interviewing techniques are vital to participants and 
should be accounted for in some reportable way. 

Reporting forms ... should cover other information and services we do for clients. 

Allow more explanation of our activities, even if it is just a couple of lines. 

Current forms do not include retention information or how many of our clients come 
through, who are still active but have completed a goal or goals. 

Report on time spent assisting a person who will never become a participant. 

I participated in the work Group that made the recent revisions... I urge that the process be 
revisited from time to time and not become a format set in stone. 

With respect to the method of reporting, DINAP has recently initiated plans to 
move to electronic reporting. Most grantees who gave their opinion apparently support 
this effort, with 32% agreeing strongly and another 47% agreeing somewhat that 
“submitting reports electronically is very convenient” (question ‘d’ in Table III-4). 
Some, in fact, seem very enthusiastic about this development: 

Shift reporting to totally electronic. 

(We) want on-line reporting. 

However, an appreciable minority expressed reservations, with 20 disagreeing 
that electronic reporting was convenient, and many others (about 36% of the grantees) 




III-13 

l 85 



declined to express an opinion (question ‘d’; see footnote to Table III-4). Many, 
perhaps, would have their misgivings relieved if additional training were offered and 
the software were easier to use: 

We still have problems... Instructions were not clear. 

I would like more information on electronic reporting. 

Recent setbacks to DINAP software have made electronic reporting not as convenient as it 

should be. 

I would like to submit my reports electronically, but it is only offered for IBM compatible 

computers. I would like to see a version of the software made for the Macintosh. 

Some on-hands assistance at the beginning would be a help. 

Finally, a few grantees voiced preference for a client-level reporting system, of 
the sort that was pilot tested several years ago: 

Recommit to a SPIR type system. 



Table III-4 



Opinions about the Reporting Process 





Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


a. The recent revisions to the reporting forms 
make them easier to use 


58.7% 


36.7% 


3.7% 


0.9% 


b. The reporting forms are still too 
complicated 


10.5 


25.7 


40.0 


23.8 


c. The Annual Status Report covers all the 
information that needs to be reported 


39.4 


47.7 


11.9 


0.9 


d. Submitting reports electronically is very 
convenient 


32.4 


47.1 


10.3 


10.3 



Note: Figures represent the percentage of grantees who gave the responses indicated, after removing 
those who gave a response of "No Opinion” from the base. Only small numbers of grantees gave an 
answer of "No Opinion,” except for item d, on which 36% of the grantees gave this response. 



Performance Standards 

Perhaps no issue has raised as much rancor among grantees over the years as 
performance standards. Grantees have argued that performance standards “drive our 
programs” and cause them to put externally imposed performance benchmarks ahead of 
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the needs of their clients. Whether these arguments are more than rhetorical is unclear; 
after all, all but a handful of grantees have exceeded their minimum performance 
standards by wide margins year after year. Indeed, the vast majority easily meet their 
standards on all three measures, even though satisfactory performance on only two of 
the three is required. 

Nonetheless, grantees do operate in extraordinary diverse circumstances, as 
Chapters I and II have made clear. Given this diversity, grantees could doubtless 
demonstrate their chief accomplishments more clearly if performance measures allowed 
for more flexibility. Moreover, grantees feel a lack of ownership of the current 
performance system, and this more than anything has bred resentment and ill will. 

The Administrator Survey asked a number of questions about performance 
standards as a way of tapping grantee sentiments on this issue. Several key themes 
were raised, including whether the current performance measurement system is clear, 
fair, and provides meaningful measures of performance. The results are presented in 
Table III-5. 

As the table shows, about 85% of grantees understand the current system 
(question ‘a’) and 75% are satisfied with it (question ‘b’). Grantees also generally feel 
that performance standards provide targets that help them improve program 
performance (question ‘c’), with over 70% agreeing with this statement. These 
sentiments are reflected in the following comments; 

The current standards are difficult to grasp at first, but are fair and feasible. 

Current standards (with narratives) are fine for us. . . Those happy with current standards 
should be able to keep them. 

I must say that I have never felt ... that the performance standards were onerous. They 
actually seem to be right on target. 

However, an appreciable minority of grantees disagreed with these three 
questionnaire items. Moreover, although the model adjustments (used to set minimum 
standards for the entered employment rate and positive termination rate) are generally 
viewed as fair (question ‘e’), they are also perceived by most grantees as difficult to 
understand (question ‘d’). More importantly, although most grantees feel that the 
current measures “give us enough flexibility in running our program” (question ‘j’), 
over 70% also feel that the measures “do not adequately reflect the goals or 
accomplishments of our program” (question ‘f’) and focus too much on job placements 
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(question ‘g’). Employability enhancements, which were introduced in PY 91 to give 
more flexibility, are generally viewed as having helped grantees focus on long-term 
training needs (question ‘h’), but they too are viewed as difficult to comprehend 
(question ‘i’). 

Thus, although the current system may be something that most grantees can live 
with and that some even prefer, sentiments are clearly ambivalent. In their comments, 
grantees gave a number of suggestions for what sorts of improvements they would like 
to see. Some comments reflected suggestions for additional measures: 

We should have periodic measurements for participants’ progress while in the program, 
prior to termination. 

We should measure some services that do not necessarily require monetary involvement. 
Update the definitions on employability enhancements. 

Expand the employability enhancement outcomes 

We need a way to claim credit for training assistance provided to our clients. 

Rather than job placement (success should be defined as having) the client complete the 
program. 

I would place more emphasis on pre-employment training. 

DOL measures quantity: it should be quality. 

(We should) terminate and re-enroll 2-year clients who are college students as completing 
successfully at the end of each year. 

Other suggestions related to the methods by which measures would be established 
and standards set. 

A menu option should provide more flexibility. 

Use more narrative. 

I believe there should be another method for smaller grantees. If one client misses a goal, 
performance standards are thrown off. 

(We want) individually designed performance measures. 

Each grantee is in a unique position. Therefore to some degree the measurement of 
performance can be done on a case-by-case basis. 

Grantees should be given the opportunity to develop (measures) in addition to those 
currently used: standards would reflect more closely the thrust of the local programming. 

Importantly, many of these comments are right in line with recommendations 
recently issued by a Work Group convened by DOL to assist in revamping the current 
performance measurement system, including its suggestion to move to a menu system, 




III-16 



88 



add measures reflecting outcomes other than employment (i.e., skills attainment and 
community outcomes), and allow for grantee-determined measures. 

Regardless of the system itself, some grantees would like additional feedback 
from DOL on whether they made their standards or not: 

(We should get a) letter or call on whether we passed or failed . 

Others don’t like the whole notion of performance standards: 

(I believe DOL should) eliminate performance standards. 

While others think they should be made more stringent: 

Set performance standards high and punish the program if we don't meet them the next year. 



Table III-5 

Opinions about Performance Standards 





Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


a. I understand the current performance 
standard system 


35.1% 


51.4% 


11.7% 


1.8% 


b. Overall, our program is satisfied with the 
current performance standards system 


21.3 


53.7 


19.4 


5.6 


c. Performance standards provide targets that 
help us improve our program performance 


21.8 


51.8 


20.0 


6.4 


d. The model adjustments DOL uses to 
establish standards are too complicated 


13.6 


59.2 


22.3 


4.9 


e. The process DOL uses to adjust 
performance standards is fair 


9.8 


59.8 


25.5 


4.9 


f. The performance measures do not 
adequately reflect the goals of our program 


22.3 


51.5 


19.4 


6.8 


g. Performance standards established by 
DOL focus too much on job placements 


25.5 


48.1 


21.7 


4.7 


h. the use of employability enhancements has 
helped us focus on longer-term training 


35.5 


49.5 


11.2 


3.7 


i. The definitions of enhancements are too 
difficult to understand 


18.2 


40.9 


31.8 


9.1 


j. the current outcomes give us enough 


21.7 


45.3 


31.1 


1.9 



flexibility in running our program 



Note: Figures represent the percentage of grantees who gave the responses indicated, after removing 
from the base the generally small numbers who gave a response of 'No Opinion. ' 'No Opinions’ 
were largest, approaching 10%, for items d, e, and f. 
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Technology 

The Department of Labor has recently initiated a strong movement to get all 
grantees hooked up electronically, to each other and the DINAP Home Page. Although 
grantees generally support these initiatives, responses included in the above sections on 
technical assistance and reporting suggest that some grantees are wary or need 
additional help. For example, we saw above that from one-quarter to about one-third 
of the grantees reserved judgement on whether they accessed the DINAP Home Page to 
get information or thought that electronic reporting was convenient. 4 Quite a few 
grantees also wrote comments that expressed a plea for more technical assistance on 
electronic issues and for user-friendly software. 

In an effort to learn more about grantees’ reactions to the technology initiative, 
we also asked a number of questions explicitly about whether grantees are accessing 
information on-line. For example, we asked if they were an “online partner” and, if 
so, whether they checked their e-mail at least weekly. As shown in Table III-6, 
responses to both these questions suggest that the technology project has made 
substantial progress. For example, over 70% of respondents indicated that they were 
online partners, and, of those who were, almost all check their e-mail frequently. 
Moreover, grantees are very receptive of the DINAP “Hot News Flash” and find the 
DINAP Home Page and Netforum very useful. 

As part of an effort to improve the electronic capabilities of grantees, DOL has 
also recently considered developing an automated Management Information System 
computer software for free distribution to grantees. As part of the survey, we asked if 
such a program would be viewed as valuable. Grantees overwhelmingly indicated that 
it would, with 75% citing it as extremely useful, and almost all others rating it as quite 
or somewhat useful. In their comments, many grantees additionally emphasized how 
valuable such a system would be, but one suggested that: 

An automated MIS should include all DOL programs 

Clearly, it seems that grantees are receptive— even enthusiastic— about DOL’s 
latest initiatives to improve the programs’ technological capabilities. However, as the 
comments on technical assistance suggested, additional training may be needed to bring 
everyone up to speed. 



4 See the notes to Tables III-2 and III-4. 
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Table III-6 

Opinions about the Technology Initiative 






Yes 


No 


Not Sure 




a. We are an “on-line” partner 


71.7% 


28.3% 


-- 




b. We check our e-mail at least once a week 


87.5% 


12.5% 


— 






Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


c. I enjoy receiving the DINAP “Hot News 
Flash” 


73.4% 


26.6% 


0.0% 


0.0% 


d. The DINAP Home Page and Netforum are 
very useful to grantees 


65.7 


32.8 


0.0 


1.5 




Extremely 

Useful 


Quite 

Useful 


Somewhat 

Useful 


Not at all 
Useful 


e. How useful would it be if DOL were to 
develop automated MIS computer software 
for free distribution to grantees 


75.5 


16.0 


7.5 


0.9 


Note: Whether grantees check their e-mail, enjoy receiving the news flash, and find the DINAP 
Home Page useful was only asked of those who were online partners. All percentages were 
calculated after removing those who gave a response of “No Opinion” or “Not Sure” from the base; 
these represent 19% for question c, 15% for question d, from among those who are on-line partners. 



Overview of Grantee Relations with DINAP 

As a way of summarizing sentiments towards DINAP, we asked grantees to rate 
the overall helpfulness of "the assistance you received from DINAP in accomplishing 
your program objectives.” These results, reported in Table III-7, show that over 80% 
gave a rating of quite or extremely helpful, and that almost no one rated DINAP as 
“not at all helpful.” 

In general, then, grantees apparently have quite a high regard for DINAP. As the 
results presented throughout this section attest, grantees are particularly appreciative of 
the recent partnership initiative. There is almost unanimous appreciation of DINAP’s 
strong efforts towards fostering good dialogue as part of the consultation process, and 
grantees believe that the recent changes in reporting and (inferentially) the 
recommendations of the Work Group on performance standards are on the mark. They 
greatly appreciate as well the strong efforts DINAP has made to see that their technical 
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assistance needs are being met, although new Directors seem to need additional 
assistance. Grantees are willing to embrace recent efforts towards the use of more 
sophisticated technology, but some are wary and will need additional training and 
guidance. 

In concluding this section, we present some additional comments provided by 
grantees in response to the query "What additional services or assistance would you 
like from DOL?” 

Provide more funding 

Provide flexibility re economic development 

Keep grantees aware of the legal aspects of all statutes and regulations of relevant agencies. 

Many suggestions are made to improve our program. But I would like to see for myself a 

program that is running “picture perfect,” to get some ideas from them. 

We would like market information for our area. 

Each grantee needs a letter of recognition each grant year if they have met the goals of the 

program. 

Table III-7 

Overall Rating of DINAP’s Helpfulness 

Overall, how helpful is the assistance you received 
from DINAP in accomplishing your program 
objectives? 



Extremely helpful 


39.4% 


Quite helpful 


41.3 


Somewhat helpful 


17.4 


Not at all helpful 


1.8 



Evaluation of Other DOL Functions 

Most of the questions and comments described above focus on relationships 
between the grantees and DINAP, the federal agency with which Section 401 programs 
are likely to have the most frequent contact. However, grantees also have dealings with 
departments or agencies within the Department of Labor other than DINAP. We asked 
grantees to rate their satisfaction with these additional DOL functions. 
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As Table III-8 shows, almost no one expresses dissatisfaction with any of these 
agencies, including the Office of Grants and Contract Management, the Division of 
Resolution and Appeals, the Closeout Unit, and the Division of Performance 
Management and Evaluation; put differently, those who express an opinion generally 
are somewhat or very satisfied. However, appreciable numbers of grantees— over half, 
in the case of the Division of Resolution and Appeals, and at least 30% in all other 
cases— express no opinion, suggesting that they either blur the distinction between 
DINAP and these other agencies, or have too few dealings with them to make an 
informed judgement. 5 



Table III-8 

Satisfaction with Other DOL Offices 





Very 

Satisfied 


Somewhat 

Satisfied 


Somewhat 

Dissatisfied 


Very 

Dissatisfied 


No 

Opinion 


Office of Grants and 


26.4 


36.4 


3.6 


1.8 


31.8 


Contract Management 


Division of Resolution and 


18.2 


18.2 


6.4 


0.9 


56.4 


Appeals 


Closeout Unit 


26.6 


19.4 


11.0 


0.9 


32.1 


Office of Performance 


23.4 


27.0 


7.2 


0.9 


41.4 



Management and Eval. 



Program Goals 

As discussed in Chapter I, Section 401 programs frequently operate in 
extraordinarily difficult circumstances, including trying to meet the needs of hard-to- 
serve program participants in often bleak socioeconomic contexts. Given these 
circumstances, they must balance diverse program objectives, from providing long-term 
training, to providing a source of income in the short-term, and from meeting the needs 
of individual participants to enhancing the well being of the community as a whole. 



5 The "No Opinion” response category was an option in most other sections of the questionnaire 
that describe grantees' sentiments towards technical assistance, consultation, reporting, and performance 
standards issues. However, so few grantees checked this response category in these other sections 
(except for the few questions dealing with technology issues) that we have not bothered to show it 
explicitly up to this point. 
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As part of the Administrator Survey, we asked grantees what they view as their 
most important program priorities. Results to these survey questions are shown in 
Table III-9. Three goals, all very much in keeping with the intent of employment and 
training programs, loom as most important. Specifically, over 75% of grantees rate 
providing training, helping participants find jobs, and helping them achieve self- 
sufficiency as extremely important to their programs. Thus, at least to this degree INA 
grantees take a very traditional view of their chief functions. 



Table III-9 
Program Goals Identified 

Extremely 

Important 


by Grantees 

Quite Somewhat 

Important Important 


Not at all 
Important 


Providing participants with training 


81.3% 


17.9% 


0.9% 


0.0% 


Helping participants find jobs 


77.9 


21.2 


0.9 


0.0 


Helping participants achieve self- 
sufficiency 


83.2 


14.2 


2.7 


0.0 


Assisting participants with their 
supportive needs 


35.1 


49.5 


15.3 


0.0 


Providing participants with 
temporary income while in 
training 


31.9 


41.6 


18.6 


8.0 


Providing a place in the community 
where people know they can come 


63.7 


27.4 


8.8 


0.0 



Doubtless unique among other JTPA programs, however, about two-thirds also 
rate providing a place in the community as extremely important. Reflecting themes that 
will emerge quite clearly in subsequent chapters as the results from the site visits are 
described, the sense of community fostered by INA grantees is very important to the 
vibrancy and effectiveness of these programs, and that fact is reflected in these survey 
findings. 

Although they are rated less highly, providing supportive services and providing 
temporary income are also viewed as important program goals. Thus, grantees find 
that not only must participants’ long-term employment and training needs be met, but 
their immediate needs for sustenance must be addressed as well. 
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Finally, grantees were asked to write in any additional goals they had established 
for themselves. Among responses they provided were: 

Crucial role of our program is improving self-esteem; this is pivotal to our cultural identity. 

Provide career exposure. 

Encourage self-sufficiency so participants can function in two worlds— traditional and main 

line society. 

Provide direction and counseling for Native families. 

Help participants find other services. 

Intervention and mediation in employee/employer matters. 

Another grantee, focusing on interests beyond those of the participant alone, 
mentioned that supporting tribal operations with subsidized job placements was also an 
important objective. 

Support tribal operations with subsidized job placements. 

This objective also emerged clearly from our on-site visits, as will be discussed in the 
chapter later in this report on work experience and community service employment. 

The survey results clearly suggest that grantees have diverse objectives in mind as 
they develop their service strategies. But, potentially, the program goals identified by 
grantees as being most important vary in important ways as a function of the contexts 
within which grantees operate. Job placements, for example, may be less important in 
service areas that offer few job opportunities, and instead training may come to the 
fore. Conversely, training may be less important for programs serving more job-ready 
clients in an active labor market, so a focus on quick job placements may become the 
priority. The complexity of program context cannot be easily characterized by any 
simple summary measure, but for analysis purposes we have crudely drawn a 
distinction between tribal and non-tribal programs. Table III- 10 thus shows which 
program goals were rated as extremely important for these two types of grantees. 

Overall, the results are surprising for how similar tribal and non-tribal programs 
view their mandates. For example, about equal percentages of both groups rate 
providing training, jobs, supportive services, and temporary income as extremely 
important; in fact, none of these differences attain statistical significance at 
conventional thresholds. Thus, if differences in the extent to which these goals are 
rated as important varies as a function of the socioeconomic context, tribal government 
status serves as a poor proxy. 
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Table IIMO 

Differences in Program Goals for Tribal 
and Non-tribal Programs 






Non-Tribal 


Tribal 


Providing participants with training 


84.2% 


80.6% 


Helping participants find jobs 


76.3 


79.5 


Helping participants achieve self- 
sufficiency 


89.5 


79.5 


Assisting participants with their 
supportive needs 


36.8 


35.2 


Providing participants with 
temporaiy income while in 
training 


36.8 


30.1 


Providing a place in the community 
where people know they can come 


79.0* 


56.2 


Note: Figures represent the percentage of non-tribal and tribal programs 
rating each program goal as “extremely important.” 

*The difference between the two groups is statistically significant at the .05 
level. 



However, the table also shows that non-tribal programs are much more likely 
than tribals to rate providing a center for the community as extremely important (79% 
vs. 56%). It appears, then, that fostering a sense of community is viewed as less 
central to the mission of tribal JTPA programs, presumably because of their natural and 
historical sense of cohesion and because so many other agencies and events already 
serve this function in the tribal context. 

Services Provided to Participants 

As a way of rounding out the in-depth observations from our site visits, we asked 
a series of questions about the services grantees provide to participants, including those 
relating to assessment and service planning and training and subsidized employment. 

We also asked them what made it most difficult to meet the needs of their clients. 

These results are summarized in this section. 

Assessment and Service Planning 

The first step in service delivery is assessing the needs of program participants for 
employment and training assistance and other support. We asked grantees in which 
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areas clients were assessed and what methods of assessment were used. Table III- 11 
shows that participants’ work history and barriers to participation are assessed almost 
for everyone. Occupational interests and skills and life skills are also usually assessed, 
and informal or mixed methods are likely to be used. Elsewhere, the table shows that 
reading skills and, less often, math skills are commonly assessed, usually (but not 
always) using formal tests of basic skills. Finally, when asked for additional areas of 
assessment, some grantees cited: 

Substance abuse 
Domestic issues 

The participants’ motivation and determination 

The overall stability of the participants’ personal situation (with respect to housing, income, 
etc.). 

In general, then, it appears that most grantees conduct a wide-ranging assessment 
for all or most clients, covering a broad range of skill areas and supportive service 
needs. Chapter V will have much more to say about the characteristics of these 
assessment efforts and their overall quality. 



Table III-ll 

Areas and Methods of Assessment 



Reading skills 
Math skills 
Occupational interests 
Occupational skills or aptitudes 
Previous work experience 
Barriers to participation 
Life skills 



For How Many Clients 


Assessed for 
All/Most 


Assessed for 
Some/None 


75.9% 


24.1% 


59.5 


40.5 


85.3 


14.7 


72.2 


27.8 


96.4 


3.6 


97.3 


2.7 


68.9 


31.1 



Using What Methods 



Formal Informal or 

Methods Mixed 



68.0% 


32.0% 


62.5 


37.5 


16.5 


83.5 


15.0 


85.0 


na 


na 


na 


na 


na 


na 



Note: The first two columns report the proportion of participants assessed in each of the areas; 
the next columns report whether formal assessment methods or informal or mixed (formal and 
informal) methods are used (with responses of “not assessed” deleted from the computations). 
Methods of assessment were not asked for previous work experience, barriers to participation, 
and life skills, because it was assumed that informal methods were used. 
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Table III-12 shows that assessment results are just one of a set of factors that are 
taken into account in developing the clients’ service plans— and in fact is among the 
least important factors. Other, more practical constraints must also come into play. 

For example, the availability of appropriate training providers is mentioned by more 
grantees as being extremely important than any other single factor, and it is rated by 
almost all grantees as being at least quite important. Participants’ own preferences are 
also quite important. However, because only two-fifths of grantees cited this factor as 
extremely important, clearly participants’ preferences are not the driving factor in 
service planning. 

Other factors cited as important are the participants’ ability to support themselves 
through training. The importance of the programs’ own inadequate levels of funding— 
which surfaced again and again during our site visits— is shown by these survey results 
to also limit the types of services the participant can access, beyond what they ideally 
might want or need. 



Table III-12 

Factors Important in Determining Services Provided 






Extremely 

Influential 


Quite 

Influential 


Somewhat 

Influential 


Not at all 
Influential 


Results of basic skills assessment 


28.4% 


45.0% 


22.0% 


4.6% 


Results of assessment of interests or 
aptitudes 


27.3 


49.1 


20.9 


2.7 


Availability of training services in 
the area 


54.1 


40.5 


5.4 


0.0 


Participant’s own preferences 


40.0 


49.1 


10.9 


0.0 


Participant’s ability to support self 
through training 


46.4 


39.1 


11.8 


2.7 


Lack of program funds to offer the 
training participant wants/needs 


45.5 


39.1 


14.5 


0.9 


Lack of child care 


37.8 


32.4 


26.1 


3.6 


Lack of transportation 


47.3 


30.0 


19.1 


3.6 




III-26 



98 



Less important— but still cited as at least quite important by most grantees— are 
the participants’ lack of access to child care or transportation. Some grantees also 
wrote-in responses that included: 

The characteristics of the local labor market 
The participant’s commitment 
The participant’s lack of self-confidence 
Problems with health or substance abuse. 

Clearly, case managers need to juggle multiple factors and constraints to arrive at a 
workable service plan for their program participants, a fact that will be made clearer in 
a subsequent chapter of this report. 

Providing Supportive Services and Stipends 

As the above results suggests, lack of child care or transportation are often 
important limitations on the types of services grantees can access. As part of their 
efforts to ensure successful outcomes for participants, INA grantees often must make 
arrangements to ensure that these limitations can be overcome. More generally, as part 
of the grant responsibilities, grantees can provide an array of supportive services to 
ameliorate so-called “barriers to employment," either in conjunction with employment 
and training assistance or as stand-alone activities. Similarly, grantees are known to 
provide stipends to help participants support themselves through training. 

However, heretofore little hard information has been available to describe the 
prevalence with which these services are offered, because, unlike most other JTPA 
programs, the reporting forms associated with Section 401 do not elicit any information 
about the extent to which supportive services or stipends are provided. As part of the 
Administrator Survey, therefore, we attempted to redress this gap. 

Table III-13 provides grantees’ responses to the query “What proportion of your 
PY 96 JTPA terminees received the following services?” As the results show, grantees 
are providing an array of supportive services, to address the often multiple needs of 
program participants. At the same time, supportive services and stipends are being 
used very judiciously, as indeed they should be. Thus, all grantees are providing 
supportive services of some type to at least some clients, but in different combinations, 
presumably depending on the clients’ individual needs. 
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Table III-13 



Proportion of Terminees Provided 
Supportive Services and Stipends 








Almost 

None 


Some 


Many 


Almost 

All 


Transportation assistance 


24.0% 


33.0% 


25.0% 


18.0% 


Health care 


70.0 


26.7 


1.1 


2.2 


Family care (inc. child care) 


37.8 


40.8 


16.3 


5.1 


Personal or family counseling 


21.3 


43.6 


17.0 


18.1 


Housing or rental assistance 


51.6 


34.4 


10.8 


3.2 


Relocation assistance 


81.1 


14.7 


4.2 


0.0 


Tools, equipment, and clothing 


15.2 


38.4 


39.4 


7.1 


Meals and other nutritional assistance 


51.6 


24.7 


19.4 


4.3 


Stipends or help in obtaining 
financial assistance 


19.4 


23.3 


30.1 


27.2 



Note: Figures represent the percentage of grantees who gave the responses indicated, after 
removing from the base those who gave a response of ‘Not Sure.’ ‘No Opinions’ were largest, 
approaching 10%, for items d, e, and f 



Of the services inquired about, transportation assistance, personal or family 
counseling, and stipends are the most commonly provided, with many or all clients 
receiving these services in 35% or more of the programs. By contrast, health care, 
housing and rental assistance, relocation assistance, and meals and nutrition assistance 
are rarely provided in more than half of the programs. 

Barriers to Having Clients Achieve Success 

Finally, we asked grantees several questions to learn more about what makes it 
difficult to achieve success with clients. Two sorts of factors were asked about: 
external characteristics, such as characteristics of the local labor market, and 
characteristics of clients themselves that make it difficult for them to achieve their 
goals. 

Table III-14, which reports these results, shows that, among external constraints, 
lack of job opportunities and lack of adequate funding loom as especially important. 
Over 50% of grantees rates these factors as very important and another one-quarter rate 
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Table IIM4 

Barriers Affecting Clients’ Success 





Very 


Quite 


Somewhat 


Not Very 


External Constraints 


Important 


Important 


Important 


Important 


Restrictions and limitations 
imposed on use of funds 


27.5% 


41.3% 


19.3% 


11.9% 


Performance standards 
requirements 


19.8 


45.9 


25.2 


9.0 


Lack of training providers in the 
area 


21.6 


33.3 


25.2 


19.8 


Lack of adequate job 
opportunities in the area 


50.5 


26.1 


10.8 


12.6 


Lack of funds to provide the 
services our participants need 

Client Characteristics 


59.6 


26.6 


10.1 


3.7 


Lack of motivation 


47.3 


34.5 


17.3 


0.9 


Don’t understand what it takes to 
get and keep a good job 


41.8 


39.1 


18.2 


0.9 


Lack reading and writing skills 


29.1 


37.3 


27.3 


6.4 


Family responsibilities 


49.5 


43.2 


6.3 


0.9 


Lack transportation 


44.1 


32.4 


16.2 


7.2 


Substance abuse 


42.7 


35.5 


18.2 


3.6 


Reluctance to pursue distant jobs 
because of community ties 


50.9 


28.2 


15.5 


5.5 


Cultural ceremonies make it 
difficult to attend trng/emplymnt 


9.0 


22.5 


33.3 


35.1 


Are gang members 


3.7 


10.2 


15.7 


70.4 


Lack work experience 


54.1 


35.1 


9.9 


0.9 


Lack necessary life skills 


36.9 


42.3 


18.0 


2.7 



Note: Figures represent the grantees’ responses to questions 31 (external constraints) and 32 
(client characteristics). Question 31 asks about the factors that make it “difficult for you to meet 
the needs of your ... participants;” Question 32 asks how important the client characteristics are 
“in explaining why some of them don’t achieve their goals while in your program.” 
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them as quite important. Restrictions on the use of funds, performance standards 
requirements, and lack of training providers in the area also are cited as quite important 
by more than half of the grantees. 

Among client characteristics, the clients’ lack of motivation is rated as important, 
as are their lack of understanding about what it takes to get a job, their family 
responsibilities, their lack of transportation, substance abuse, reluctance to relocate to 
accept a job, and their lack of work experience. Other responses written in by grantees 
as participants’ barriers to successful achievements include these reasons: 

Clients lack self esteem. 

Long-term public assistance. 

Racial issues. 

Attitudes. 

Lack of dependable child care. 

Lack of transportation. 

Participants don’t communicate well with those outside the Native American community. 

Cultural differences. 

Attending school and trying to hold a job at the same time. 

These factors round out the picture of client characteristics as tabulated from the 
Annual Status Reports, shown in Appendix B. 

Final Comments 

In general, we have seen that grantees enjoy a relatively good relationship with 
the U. S. Department of Labor. They especially embrace the recent partnership 
initiative and related efforts being undertaken by DINAP to improve communication 
between grantees and the federal government and involve grantees in decision-making. 
They feel that DINAP expresses program goals clearly and that these goals allow them 
enough flexibility to serve their communities. Technical assistance needs are generally 
being met, and the national conferences, one-on-one meetings with Fed Reps, and peer- 
to-peer exchanges come in for particularly high praise; by contrast, additional efforts 
are necessary to meet the needs of new Directors and to ensure that all grantees get the 
assistance they need to become electronically connected. 

Reporting and performance standards are topics about which some grantees are 
seeking additional improvements. Although by no means is dissatisfaction widespread, 
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some grantees believe that the ASR omits some important information. Additionally, 
grantees would like to see more flexibility in a performance-standards system that 
would enable them to show their unique program accomplishments more clearly. 

Grantees have also indicated how extraordinarily complex their challenges are. 
Program goals are multi-faceted and sometimes work at cross-purposes. In meeting the 
needs of their clients, assessment results are only one of a number of factors that must 
be considered; very real constraints include the absence of adequate funding, the lack of 
appropriate training providers, and the difficulty many participants have in supporting 
themselves through training. Additional factors making it difficult for some clients to 
achieve their goals are the lack of job opportunities in the area and participants’ own 
weak motivation, problems with substance abuse, poor work histories, lack of 
appreciation of what it takes to get and keep a good job, and their reluctance to leave 
their communities in pursuit of job opportunities. 

In concluding the survey, we gave grantees the opportunity to make comments on 
any other issue of concern to them. Some of the comments they provided have been 
related in appropriate other sections of this chapter, if the comments pertained (for 
example) to relationships with DINAP, performance standards, or reporting. We 
present below miscellaneous other comments: 

We need more funding! 

(With the funds I have) I can only serve 25% of the people that apply to our agency. 

Please explain why we get so little, but are expected to do so much. 

Want additional funds for education and training. We could be much more effective in 
helping tribal members achieve self-sufficiency if we could receive additional funding.... The 
ratio of available money to available participants is 2 to 10. 

I feel that the Title IV-A and II-B programs have helped the participants. I just pray that 
there will be no more budget cuts, because my budget is so small as it is. 

There should be only one Federal agency administering the funds allocated to Indian tribes, 
and the tribes themselves should be the sole authority on who is served by these dollars. 

Provide notification of E&T related conferences, whether DOL sponsored or not; more 
specific training. 

I need a computer expert in DC available all the time for consultation. 

DOL needs to take a better look at the quality of some programs. 

It would be helpful to have general guidelines that all grantees can follow. Federal 
regulations are too broad — which is good, because it allows tribes to design programs — but it 
would be nice to have standardized forms and policies/procedures, form the very simple to 
the complex. 
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The greatest obstacle facing Indian Country in general is the lack of employment 
opportunities within our communities. We can train individuals, but the result is always the 
same... the training to promote self sufficiency is all well and good, but if the (jobs aren’t 
there) I see the outcome as superficial unless we get some concrete job developments in our 
area and throughout Indian Country. 

With the partnership effort DINAP has made some tremendous improvements and I 
commend Mr. Thomas Dowd for his leadership. 

One of the most important things about our program is the extensive network of partners we 
have established over the years. 

The strength of the program ultimately lies in two areas: the grantee organization, and the 
program staff. Everything should be done to provide both with all the tools necessary for 
running successful programs. Efforts should always be enhanced to provide information, 
training , and support to the front line. Much is already being done, but even more is 
needed. One suggestion is to support the grantee organization to ensure its overall 
effectiveness. We must involve the organization, not just the program. Program staff 
cannot always be counted on to carry this out. Grantee Boards and officials should be 
involved in major issues involving JTPA. 

Gaming creates jobs. Jobs creates income for families. Does increased income lead to 
increased ability, or remove families from the lifestyles learned pre-gaming? Section 401 
should allow for working with persons with identifiable problems, even though household 
incomes exceed ... low-income levels (so we can serve) youth of the above households. 

I wish we could use Title II funds with Section 401 funds when needed. 

DINAP as a whole has improved. The overall grantee/agency relationship is that of a joint 
effort and no longer a dictatorship. 

The unemployment rate (in our community) is ... very high, so it is very difficult to place 
participants in permanent steady jobs after they are terminated from JTPA. 

DINAP needs to really listen to the small, reservation grantees. 

Understanding by Fed Reps and DINAP that not all reservations are located close to urban 
areas and that achieving some goals are difficult. Not all participants are willing to leave the 
reservation and are content remaining in the local community, which makes it difficult to 
provide services. Despite these difficulties, JTPA is still trying to work with individuals 
needing our assistance. 

Develop a self-check or self-evaluation program on the computer, so grantees can answer 
evaluation questions that will be read and commented on by Fed Reps. 

If a grantee is meeting or exceeding their performance standards and has proven themselves 
to be effective in accomplishing the goals of the program, why do they need to compete for 
the JTPA grant every two years and go through the “notice of intent” and designation 
process? The designation process needs to be reviewed and revised to be fairer to current 
grantees. 

Allow JTPA programs in small, rural isolated areas with at times limited employment 
opportunities a higher dollar amount than $4000 per participant. And of course we could 
use more money due to the change in the welfare reform. Our office has been flooded with 
phone calls. 
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Continue to do the good work you are doing... maintaining open communication with 
grantees 

Our clients do not like to be in classroom training. We tend to lose out on a lot of clients 
because they do not like to be in class. 

Off-reservation/non-tribal programs like ours struggle with funding resources... our funds 
are stretched so tight because we don’t have access to the same support systems as the 
tribes. 

DOL should give serious consideration to equity of service and dollars between reservation 
and off-reservation programs. For example, youth employment funds not go only to tribes 
and welfare to work funds go only to tribes. The fact is, the majority of Indians live in 
urban areas. Local providers of service very seldom include Indians in their target 
populations to be served. 

The LLSIL guidelines are too low, especially with those families with children. 

Really enjoy the Friday Report. This should go to all grantees, whether their grant has the 
money or not. 

Our program is doing fine. We need this program, especially classroom training for the 
long-term. Our participants want to be self-sufficient-a real skill with a real future. That’s 
what the program is all about. 

As a community agency, we have problems meeting the expectations of some of our 
community members. Members have multiple problems and poor skills and want work, not 
training . Once clients terminate, they often return and expect lifelong services. Mentally 
disturbed clients have few places to receive help, or will not accept help. 

To break the cycle in urban country we have to give back to our youth their identity of self. 
For our youth, we need to teach them their own tribal culture and give them activities, 
sports, etc. Unfortunately, no one funds these programs. 

Good job, thank you. 
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IV. GRANTEE ORGANIZATION 
AND SERVICE STRATEGIES 



Goals and Objectives 

Going well beyond the snapshots provided by the Administrator Survey, this 
chapter describes the stated goals and objectives of the 23 IN A 401 grantees that we 
visited on site and draws upon various documents provided to on-site researchers such 
as brochures, pamphlets, reports and Comprehensive Annual Plans. Where the goals 
and objectives were provided to us in written form, we were able to review the 
wording verbatim and conduct a key word content analysis. We further incorporate the 
paraphrasing of goals and objectives as they were voiced by program administrators 
and personnel who responded to us during our on-site interviews. No attempt is made 
to separate the written from the spoken, overall, but conflicting perceptions are noted 
when they occur. 

This section incorporates both the general over-reaching goals of the twenty-three 
grantee tribes and organizations as well as the more specific goals and objectives 
described for their INA 401 programs. These two sets are somewhat blended together, 
with wording and concepts which generally coincide and link INA 401 goals and 
objectives to the grantees' organizational/tribal goals. 

Before going into the analysis of content, we need to clarify the concepts of goals 
and objectives as they are operationalized in these programs. First, the objectives are 
not technically measurable. That is, they, unlike measurable objectives in some other 
federal programs, do not contain baseline participant levels of ability or levels of 
service nor a concomitant projection of the levels of success or attainment to be 
reached. They do not serve as predictors of program success except in a general way 
using words such as: to enhance, to increase, to provide, etc., nor do they provide a 
basis upon which to evaluate programs or particular services' effectiveness in any 
measurable way. 

They do provide guidelines or frameworks. They are reflective of underlying 
beliefs of the tribe or organization and reflective of what the grantees perceive the 
function of the INA 401 program to be. They are phrased in positive language and 
often reflect the grantees' philosophy of service and purpose. 
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As to structure and specificity, goal statements and objective statements are used 
interchangeably. What one grantee may declare as a goal, another may state as an 
objective and vice versa. This is also true for statements of mission or purpose, 
whether they are organizational or programmatic. 

If goals and objectives are meant to play a meaningful role in the design and 
implementation of program services, grantees should receive training in the 
construction of meaningful measurable objectives. However, since performance 
standards are currently the formula-driven quantitative measurements required to judge 
"success", there is a gap overall in reporting and evaluating the full impact of services 
upon participants. 

Were grantees to write client-centered measurable objectives which match their 
services, they could have a framework for setting up processes to project, track and 
assess levels of skill/educational attainment and degrees of growth in knowledge and 
behaviors. This would give a broader picture of program results which would go 
beyond the categories of "terminations", would address qualitative as well as 
quantitative outcomes, and would be more inclusive of all of the actual services 
rendered to clients. 

Samples of INA 401 program goals and objectives vary, ranging from minimal 
statements such as "formed for the purpose of providing employment and training to 
tribal members" to one-stop inspired language such as "creating a customer-driven 
system of integrated services to which there is universal access and measurable 
outcomes", to an extensive listing of objectives infused with culturally-relevant 
overtones. An example of this is "to work within cultural values and beliefs, both 
traditional and modern." 

Taking a closer look at structure and specificity of INA goals and objectives, a 
few grantees only make perfunctory general statements or those which only restate the 
INA 401 regulatory purpose. They are developed to comply with the expectations of 
the funding agency and use such language as "the (INA JTPA) program will affirm the 
goals set forth by DOL" . Of the 23 grantees in our sample, six or 26% did not go 
beyond the required basic statements) of service provision, to our knowledge. 

The remaining 74% designed goals and objectives that were broader and more 
detailed, and incorporated and reflected the local community or tribal mission, purpose 
or underlying philosophy. These varied somewhat in terms of length and specificity 
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but, in general, the grantees made statements which defined their program purpose, 
listed services offered, noted clients to be served, and stated generally expected 
outcomes. 

A key word analysis of concepts which were used in the phrasing of goals and 
objectives provides us with one way to determine the emphasis grantees place upon 
programs, the variations, and the underlying purpose and expectations for program 
performance. 

The following categories describe, in descending order, the frequency with which 
grantees construct concepts into goals and objectives. We may surmise, in turn, that 
the more frequently the words or concepts are found, the more importance is placed on 
them throughout the sample. 

1. Services - "Services" of the JTPA program were referred to approximately 49 
times throughout the goals and objectives, giving an indication that the program is, first 
of all, perceived to be a service-oriented program and that its main purpose is to 
provide services to its clients. While this may seem an obvious point, it is, in the 
context of being a program operated by American Indian entities, meaningful in that it 
coincides with the cultural value of "helping/serving others" and the expectation that 
these programs will truly be of benefit to the clients. The INA program staff with 
whom we visited with few exceptions exemplified and projected this belief. Included 
in the service category were: 1) references to services in general (9 times), and 2) 
references to specific services. Specific services included: Assessment and Placement 
(1 1 times) ; GED/ABE (7 times) , Guidance and Counseling (7 times) , Supportive 
Services (4 times) , Careers (3 times) , Literacy (2 times) , Referrals (2 times) , and Work 
Experiences and Voc-Tech each one time. The emphasis on "Service to Others" may 
also explain why it is, in the minds of some grantees, sometimes problematic to meet 
the needs of the clients when the program regulations or grantee policies do not provide 
the flexibility to do so. 

Training and Employment Services is a category by itself and is addressed further 
on in this section. With that in mind, if the two were combined, "services" is the 
overriding concept stated in grantees' goals and objectives. 

2. Outcomes - Some mention of positive "outcomes" as the result of the INA 401 
services to participants was the second most frequently mentioned concept, occurring 
approximately 38 times throughout. This seems to suggest that "services will bring 
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about positive results" is the operating expectation. As one might expect, those results 
are seen in terms of "skills attainment", "employability", "job placement", "increased 
earnings" and leading to unsubsidized employment, all of which suit the purposes of 
the DOL/INA 401 program. However, the goals (and sometimes the objectives as 
well) purport to go beyond "employability" and include self-sufficiency, well-being, 
quality of life, self-direction, self esteem, success and empowerment. A few grantees 
also included sovereignty (tribal) and self-determination, as over-riding goals as well as 
linking them to program goals. 

The INA 401 programs are, therefore, seen by grantees as more than employment 
and training programs. Tribally-operated 401 programs are a part of the whole— that 
is, one of multiple services which the tribes operate for the benefit of their members 
and for which the members hold the tribal governments and programs responsible. 
Urban-based INA 401 programs goals and objectives do not reflect the "blanket of 
service" concept as strongly, but, where multiple services are provided, the same 
philosophy is reflected. 

3. Clients - As the next most frequently mentioned, "clients" or participants are 
noted approximately 30 times throughout. In these instances the grantees included the 
clients or participants as central to the formulation of goals and objectives, not just 
focusing on the program services alone. In so doing, most emphasized that the purpose 
of their INA 401 programs was to serve or provide services to special populations 
which included the unemployed, under employed, economically disadvantaged, at-risk 
and/or hard to reach. Nine of the twenty-three sampled sites specifically mentioned the 
above categories and six specifically mentioned dropouts as inferred targets for 
services. Extending beyond the clients themselves, family and community were 
incorporated four times into goals and objectives. Three grantees referred to 
"individuals" or "personal" needs and benefits as well as client groups and two 
indicated that "getting people off public assistance" was part of their objective. 

While addressing special populations is not unique to JTPA programs, the 
inclusion of family and community in tribal and urban organizational goals and 
objectives indicates a purpose more widely conceived and recognizes that the programs 
do impact families and communities as well as individual participants. 

4. Training Assistance - Mentioned as frequently as clients was preparing clients 
to be employable (approximately 29 references) . This generally seemed to include 
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what is understood to be Training Assistance, such as instruction or counseling in areas 
including "entry into the workforce," "how to keep a job," "workplace maturity", 
"ethics", "dress and behavior on the job", "motivation", and other employability skills 
or behaviors. To some, it also carries the meaning of skills assessment, career 
searching, resume writing and other attendant topics which are not thought of 
specifically as classroom training for occupational skills or basic skills instruction. 
These we are grouping separately for purposes of analysis and contrast. 

If one draws conclusions based on this analysis of frequency, Training Assistance 
is the most emphasized type of service and draws upon the premise that the special 
populations of INA 401 clients lack work place experience as one important factor in 
their unemployment or underemployment. 

In this category are also those goals and objectives that specifically mention 
employment as the intended end result of program services and a primary program 
objective. This concept occurred 17 times in goals and objectives and by 
approximately the same number of grantees, indicating a substantial emphasis being 
placed on employment. This, no doubt, is driven by regulatory and performance- 
standards requirements but may also be indicative of the need to do "all that can be 
done" to reduce the high unemployment rates of clients in isolated tribal programs 
where the demand for jobs is greater than the supply. It also suggests that employment 
is seen as the most viable means to subsistence and survival to those who are 
economically disadvantaged, giving rise to the tendency to respond to these clients' 
short-term rather than long-term goals. 

A later section of this chapter will provide a more specific analysis of service 
emphasis among the grantees. 

5. Descriptors - This category contains those words that describe the nature or 
quality of the services or service design. Such descriptors are included in grantees' 
goals and objectives approximately 20 times. They are indicative of the overall 
functioning of and between services and use terms such as "coordinated", "working 
collaboratively” or "in partnership with others" (8 times). They also describe 
approaches as being "comprehensive", "holistic” or "integrated" (9 times). In this 
vein, three make mention of the "one-stop" service approach. There is a tendency to 
include this language whether or not the reality is being accomplished. For example, 
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not all grantees are always successful in collaborating or forming partnerships, as is 
explained in other chapters of this report. 

Most interesting is the "holistic" idea that seems to coincide with cultural values 
and current one-stop thinking at the same time. It is noticeable that those grantees who 
are moving most rapidly towards one-stop services are also those who include one-stop 
concepts in their goals and objectives. 

6. Education/Training - Specific mention of education or training as a goal or 
objective occurred 19 times. References to "education" of clients is used as a broader 
holistic type of goal while "training" seems to be more descriptive of activities and 
services. "Education of (their) citizens/members" is one of the overriding goals found 
in tribal Education and Training Department goals that were provided to us and is then 
linked to the training goals or objectives of the INA 401 programs. We do not here 
attempt to explain definitively what is meant by "training" but assume it includes both 
the formal classroom training for basic skills and occupational skills as well as training 
received in job-related Work Experience, Community Service and On-the-job Training. 

This category and the Training Assistance category are not therefore mutually 
exclusive. Taken on the whole, however, the training related to short-term 
employment seems to have a somewhat greater emphasis than those related to long-term 
employment. 

7. Culture - One might expect to find mention of culture more frequently than 
the eleven instances found in the goals and objectives. The word itself is not found in 
all samples of goals and objectives, but other words are also used that are culturally or 
historically related, such as "survival", "sovereignty", "holistic", "racial attitudes" and 
"ethnic identity" as well as references to tribal members, Indian people and 
communities that connect the INA 401 programs to the cultural goals of the tribal or 
organizational grantees. Extending the idea to incorporate "contemporary tribal life" as 
a part of "culture", we can also combine the last category, economy, with this one and 
see a larger inferred importance of culture beyond the obvious one. 

8. Economic - This last category, "economic," refers to mentions made about the 
local or tribal economy, about tribal economic development and about economic self- 
sufficiency. In essence, the economy of the whole and the economy of the individual 
are seen as one entity. The economic concept is linked not only to the negative 
conditions of poverty and joblessness but also to the positive aspirations of employment 
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and self-sufficiency. As other parts of this report describe, the economic conditions of 
the service areas of IN A 401 grantees play a major role in the design and functioning of 
the INA 401 programs. Specific mention of "economy" or "economic" is made nine 
times through the grantees' goals and objectives. 

In summary, grantees' goals and objectives have key elements in common but are 
very diverse in construction and format. They serve the purpose of putting general 
objectives in writing but, with few exceptions, are not stated in measurable terms and 
are therefore not conducive to forming evaluative judgements as to participant progress 
over any given performance period. 

Amounts and Sources of Funding 

The twenty-three selected projects in this study have Section 401 budgets that 
range from a high of about $7,000,000 to a low of barely $50,000. The majority of 
the sampled projects operate their Section 401 program solely on the DOL-JTPA grant, 
but, depending on the grantee, whether tribe or Indian organization, many also have a 
number of other resources on which to draw to assist potential and actual JTPA 
participants. The majority of the projects in the study attempt the "one stop" service 
delivery because they recognize their clients have needs that supersede the basic 
services of the JTPA program. Housing, food, health care, transportation and 
childcare are "other" areas of need. The JTPA-401 grantees have utilized a variety of 
strategies to meet these needs. Tribal JTPA grantees utilize those resources more 
readily available to them, such as: Department of Interior-related programs, Adult 
Education, Tribal Employment Rights Office, Department of Health & Human 
Services, Indian Health Service, Department of Commerce, and Department of 
Housing and Urban Development. These resource services are coordinated with state 
agencies in some instances but not necessarily with funds. State employment agencies 
are helpful via the JOBS programs as far as referrals are concerned, but the tribes have 
a strong stance in regard to their sovereignty and are cautious when coordinating with 
states. 

Tribally Operated JTPA Programs 

As mentioned previously, the tribal JTPA programs comply with the overall goals 
and objectives of the DOL-JTPA requirements and design their programs in the best 
ways that "fit" their tribal members' needs. Many tribes contribute "in kind" to their 
INA 401 programs by providing office space, furniture, utilities, and, in some 
instances, fringe benefits. In those instances where the tribes are isolated and few 
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opportunities exist for employment, some tribes supplement (if resources are available) 
the JTPA participants' income when their eligibility for JTPA services has reached its 
limit. For example, one large tribe has employment as a priority and contributes its 
own money to extend JTPA services. Several medium-sized tribal JTPA programs 
augment the Title II-B, Summer Youth Program, with their own funds and coordinate 
other tribal resources to provide their tribal youth work experiences. These tribes 
recognize the value of the summer youth program to provide positive activities, in 
attempts to deter negative influences on tribal youth that may prevail if they are not 
actively employed. 

Smaller tribes have invested their JTPA funds in personnel in order to provide 
JTPA applicants with a human resource that has the ability to refer, network and 
coordinate a number of available resources that the applicant may need. These tribal 
JTPA personnel use formal and informal networks to provide needed services. The 
formal network of resources these personnel use may range from the "routine" JTPA 
services: OJT, WEX, CRT, CSE to connecting to tribal services to accessing JOBS, 
BIA, etc. The informal network could range from knowing someone who works for 
tribal housing to secure housing or knowing someone in social services who can 
provide childcare. This investment of small tribal JTPA project funds on personnel 
may appear, on first glance, to be providing employment for a few people, but in 
retrospect may be a quality investment that allows the small tribal JTPA program to 
maximize its budget by employing knowledgeable people who can access many 
resources and thus serve the client in a variety of ways. 

This study has also revealed the age-old problem of tribal-inter-organizational - 
personal politics. While many tribes in this study have "stable" governments, there are 
changes every several years in the leadership of these tribes. These leadership changes, 
at times, affect the direction, personnel and services of the JTPA programs as well as 
the operation of other tribal programs. For example, one grantee that we observed 
operates its JTPA program in isolation from other tribal programs even though they are 
housed in the same building. Tribal political leadership changes may influence, who, 
how and what JTPA services are rendered. For example, Program Directors are 
changed and JTPA staff become divided on tribal issues when tribal elections change 
heads. This is a sticky issue and one not easily addressed, but it does occur. The 
conflicts that arise in the “competition” between tribal programs could cause delivery 
of services independent from one program to another. 
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This study has found that extreme diversity exists between small tribal JTPA 
programs and large JTPA programs in regards to budget and accessing "other" tribal 
programs. Factors that influence these two entities are the tribal administrative 
structure and other available tribal resources. Large tribal JTPA programs have larger 
budgets, larger populations to serve, larger services areas and larger "other" resources 
on which to draw. However, larger doesn't always mean better especially when the 
typical barriers to employment still exist: fewer opportunities for employment, fewer 
opportunities for classroom training, and lack of transportation to access JTPA 
services. The problems of serving clients in large tribal JTPA programs are 
compounded almost exponentially, along with the difference in dollars received as 
compared to small/medium-funded tribal JTPA programs. 

Indian Organization JTPA Programs 

Indian organization sponsored JTPA programs' budgets range in size from $2.37 
million to $65,626. Many of these organizational JTPA programs are located in urban 
areas. These projects have goals and objectives that conform to DOL guidelines, and 
services are provided similarly. A remarkable quality of these urban-based projects is 
their ability to access resources from a variety of entities to provide "one-stop" services 
to their clients. Though conscious of tribal sovereignty, the urban projects aren't as 
compelled as tribal JTPA projects to ensure that their tribal jurisdiction not be 
compromised by interaction or cooperation with municipal or state governments and 
private social service providers. They have successfully garnered funds to their 
organizations from agencies such as: State Drug-Alcohol Abuse programs; AIDS-HIV 
Awareness and Prevention; private foundations; State Department of Health; Office of 
Economic Opportunity; Department of Health and Human Services, Administration on 
Aging; Department of Agriculture; Community Development Block Grants; and city 
and county social-health-education services. Local private industry has also partnered 
with many organizational 401 projects, and is contracted to conduct projects related to 
that industry; this in turn provides income for INA 401 grantee organization. 

Organizational JTPA programs face many of the same problems tribal JTPA 
programs face— isolation (social-cultural) , large service area (many in large urban 
areas) , accessibility (transportation) , hostile environments (participants alienated from 
society), and visibility/communications (small populations/amalgamated with other 
minorities) . This is the motivation for the organizational JTPA programs to attempt to 
be "one stop" in the provision of services. Urban JTPA clients may be desperate for 
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help and the Indian organization may be their only life line. If the sponsoring 
organizational JTPA project can access related social services from other agencies, they 
stand a better chance to help their Indian clients and achieve their JTPA goals and 
objectives with participants. If participants' immediate needs can be addressed 
(housing, health, food, clothing, child care) then they may be better able to participate 
in Classroom Training, Work Experience, Community Service, On-The-Job Training 
or Summer Youth programs. 

It is amazing the networks that these organizational JTPA programs secure to 
meet their clients' needs. Their proximity to the various resources allows them access 
to these programs that may not readily be available to most tribal JTPA programs. 

JTPA Budget, Reduction Impact 

DOL budget reductions, without exception, negatively effected all projects in this 
study over the past two years (PY 95 - PY 96). JTPA program services of the 23 
programs in this study had to be curtailed: some participant services were ceased at 
mid-year, a dollar cap on per participant services was imposed by several projects 
($175/each, $10,000/each), employees were terminated, consolidation of JTPA offices 
occurred, workload increased for staff, fewer coordination meetings were held, and an 
overall reduction of services occurred. Some tribal and organizational programs 
supplemented the JTPA program funds and services but most did not have the resources 
and attempted to continue their programs on a restricted basis. Needed participant 
services exceeded the available JTPA resources, and projects with limited resources 
were hard pressed to meet their clients' needs. The organizational JTPA programs 
appeared better able to meet client needs due to the variety of social-health-housing 
resources available in their service areas (mostly urban) . These organizational entities 
for the most part because of their proximity to municipal, county and state resources, 
were able to access these resources and provide a variety of services not readily 
available to tribally sponsored JTPA programs. Some tribally sponsored JTPA 
programs are able to supplement their JTPA programs if they have economic 
development or other tribal projects in progress. For some tribes, casinos have 
provided the impetus to develop a variety of tribal projects that otherwise may take 
years to initiate via the Bureau of Indian Affairs, other tribal resources, or private 
investments. However, the tightening of the federal budget has placed needy tribal 
people more at risk, because of limited JTPA services in areas that already have few 
employment opportunities. Regardless of the opportunities for training for JTPA 
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participants, the prospects for employment are often dim. The budget reductions 
compound the problems faced by INA JTPA projects in this study. 

Coordination 

In order to meet JTPA client needs, tribal and organizational JTPA INA 401 
programs must draw on a variety of resources. All projects— large, medium, and 
small — have established a network of resources on which to draw, refer, and place 
JTPA participants. Those JTPA 401 programs serving urban areas have more 
opportunities to access resources that serve all people than do the tribally-sponsored 
programs, simply because they are in close proximity to those services. Conversely, 
tribal 401 programs have access to services available to tribes that may not be available 
to more urban projects, although they may have access to some of these non-tribal 
services also. 

The state employment service and JOBS program are likely components that are 
available to 401 projects. Likewise, local school districts (for GED, Basic Skills) and 
higher education institutions, both public and tribally controlled colleges, are accessed 
for CRT. Many of the 401 programs coordinate with Local/State/Federal Drug and 
Alcohol Abuse/Prevention programs. Childcare appears to be of concern to JTPA 
participants and, as such, most 401 programs attempt to access this resource through 
the tribe or sponsoring agency (e.g., Head Start). Some states have Indian Affairs 
offices that may assist the 401 programs in becoming aware of state social services, 
education opportunities, health service, child care, vocational rehabilitation, veteran 
affairs and employment and training services. These state Indian Affairs offices keep 
the Indian communities abreast of pending legislation that may directly effect them and 
also serve as advocates for tribal communities in their respective states. 

Tribally sponsored 401 programs are acutely aware of the need to maintain their 
sovereign status and are vigilant in maintaining this status in their interactions with state 
agencies. Many states throughout the USA seek to impose their jurisdiction over tribes 
to regulate how the tribes operate within those states. However, United States Supreme 
Court decisions have upheld the sovereignty of tribes and maintain their status of 
equality to states. Some tribes are so large that their 401 programs have huge budgets, 
intricate administrative structures, huge service areas and must "deal" with an overall 
tribal bureaucracy that is on the scale of a state bureaucracy. One tribal sponsored 401 
program of medium size has a 13 county area with several large metropolitan cities 
within its service areas. The size of the service area, scattered location of clients' 
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residences, and isolation (social-cultural) make for employment and training barriers 
that are most complex. These factors combined make it necessary for 401 projects to 
be creative, innovative and persistent to meet the needs of their clients. The barriers of 
few employment opportunities, transportation problems, some social alienation and 
budget reductions (i.e., reduced services) combined cause the frustration for JTPA 
participants and JTPA programs in being able to serve clients. Thus, coordination of 
available resources is paramount to 401 projects. 

Regardless, 401 projects in this study have done a remarkable job of identifying 
resources in their service area and accessing them. Arts, cultural and museum 
organizations, tourism, gaming and athletic (running) organizations, Indian Health 
Service, programs for the elderly, childcare, and domestic violence programs funded 
by city, state and federal auspices are a few “other” resources accessed to serve JTPA 
participants. If fewer resources are available, the 401 projects access those few, such 
as the county court systems for WEX, regional coordinating council/economic 
development for placement leads, temporary job services, community centers, energy 
assistance programs, tribal newspapers, state private industry councils, and veteran 
programs. One 401 program lists 27 agencies with which they coordinate as an 
organization to serve their native communities. Those small 401 projects, who invest 
in JTPA staff to provide services/referrals/assistance, utilize the staff's experience and 
knowledge of local conditions to access employment, training and social services for 
their clients. Some 401 grantees’ sponsoring agencies are more aggressive than others 
in pursuing organizational funds. Many tribes are initiating or pursuing economic 
development projects that have or will have an impact on employment opportunities, 
should they be successful. The 401 projects' sponsoring agencies often submit 
proposals to various federal and private entities for projects that may directly provide 
employment, training or on-the-job opportunities for JTPA participants. However, the 
ebb and flow of funding for these agencies (recently ebb) have limited a lot of these 
economic development proposals to not be funded. 

In summary, 401 budget allocations and coordination of services appear to reflect 
the needs of clients and available resources and are directly related to the goals of the 
sponsoring organization and the motivation (aggressiveness), experience, and creativity 
of those who work in the projects. Tribally sponsored JTPA programs, in their efforts 
to economically develop their tribal lands, face monumental tasks securing funds for 
these projects while striving to maintain their sovereign status. Indian organizationally- 



sponsored 401 programs face a similar dilemma in that their Indian constituents are 
dispersed over a large area, are as "at risk" as their reservation/rural based kin and 
many fall through the cracks due to their invisibility in the urban areas. 

Coordination of services is key to 401 sponsoring agencies’ ability to meet the 
needs of the Indian community. A thread that emerges and is consistently voiced by 
JTPA participants is that they choose and feel more comfortable working with other 
Indians. Although they are eligible for services from any public agency, 401-JTPA 
participants at times feel alienated from those agencies. Section 401 grantee agencies 
try to be a one-stop operation; some have reached that goal, others are still in pursuit. 

Staffing 

Staffing for 401 JTPA grantees in this study range from a high of 106, to a low 
of 1. Depending on the size of the project, there may be field offices that have 
anywhere from 3-5 staff persons. For example, one large tribal JTPA Program has one 
main JTPA office at the tribal headquarters and five field offices and nine sub offices 
on a reservation that covers 4 states. A large urban 401 grantee has 10 field offices 
that cover 15 counties. A tribal grantee JTPA program covers almost one half of the 
state (22 counties), with its main office on the reservation and field offices in three 
large metropolitan areas. A large tribal grantee has a main office at the tribal 
headquarters and 12 field offices to provide services in a 14 county area. The 
remaining grantees in this study have 401 projects in one location in their service area. 

Staff Positions 

Program staff includes Project Directors or Managers, Project Assistants, and 
Other Staff. Their major roles are described below. 

Project Directors/Managers. All 401 programs in this study had a Project 
Director, Executive Director or Program Manager with primary responsibility for 
overseeing the implementation of the JTPA program. Some primary JTPA 
administrators had multiple-responsibilities with the sponsoring agency, such as: TERO 
Director, Executive Director of the sponsoring organization, or coordinator of several 
tribal programs. Most JTPA Directors were experienced and long term in their 
positions. For example, some primary Project Directors are carried over from the 
CETA project times and are well aware of the history and progress of the 401 JTPA 
program. These long-term Project Directors are able to maneuver in the political arena 
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of tribal and organizational worlds to assure consistent and effective services to their 
JTPA clients. Some are more adept at these maneuvers than others. 

Some Directors are less experienced. One Project Director left "under a cloud" 

-- records were missing, and the new JTPA Project Director and staff had a difficult 
time in reassembling the records. Another exception was a Project Director who was 
in her first year; however, she had participated in the JTPA program several years 
before and had personal qualities that enabled her to administer the program well and 
continue services on the same plane as before. 

Project Assistants. Many 401 projects have JTPA project assistants, or assistant 
managers, who in many ways conduct the baseline management of the project. They 
oversee or conduct client assessment, intake and placement. They assure that all 
aspects of their JTPA programs operate in conformance with DOL requirements and 
maintain communications among the field offices and the main office. They serve as 
liaisons between administration and the day to day service providers and clients. 

Other Professional Staff. The day to day "on the street" staff— intake 
personnel, counselors, and directors of placement— provide the bulk of the support for 
services of the JTPA programs. Depending on the size of the projects, these staff 
persons perform multiple tasks and are cross-trained to be able to assure clients' needs 
are met, even if one of the designated staff for a certain task is absent or a position is 
vacant. They use their people skills and knowledge of tribe’s culture and language and 
service area/environment to make clients feel comfortable in their participation in the 
JTPA program. Comfortable in this sense means being caring, listening, being 
supportive and serving as an advocate. Many of these staff persons have participated in 
various JTPA services themselves and can be empathetic in assisting JTPA clients in 
many ways. 

Concern for Clients 

Each of the 23 programs in this study had a genuine interest and commitment to 
provide their clients appropriate and effective services. The 401 project staff 
understand the need for "Indian people to help other Indian people". They understand 
that for a lot of Indian people (clients) it may be difficult to seek assistance from non- 
Indian people. As such, a premium is placed on respecting the individual, meeting 
their needs in an unobtrusive way, and trying to anticipate their needs in an unfamiliar 
(new) environment. Provision of JTPA services in some projects emphasizes 
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consideration of culture, considerations of language and, in most cases, consideration 
of the culture of poverty. The term "family" surfaced time and again in reference to 
grantee staff. "Family" may be described as working well together, sharing 
information and cooperating well with fellow JTPA staff persons. There are exceptions 
of course, but the concept of family was generally quite strong. 

Employment Practices 

Most of the 401 JTPA programs practiced Indian preference in hiring staff. 

Those non-Indian staff employed by various programs had similar commitments to 
serve Indian people as the Indian staff. They were accepted by the Indian staff, and 
many proved their commitment by modeling their actions/interactions with tribal people 
in many and varied ways. Several projects (informally) practiced tribal preference in 
their hiring. Although not stated, these projects sought tribal people to staff their 
projects for ethnocentric purposes, or were under the impression that tribal people 
knew their own people better and could serve them better. 

Summary 

In summary, most Section 401 grantee staff are long-term employees, understand 
the JTPA program and its intent, have experience in working with their clientele and 
employers, and have a wide network of resources upon which to draw. The fact that 
most grantee staff are long term meant that they could provide consistency and 
institutional memory to the projects, although long tenure could also cause 
complacency and the routinization of services. By contrast, some grantees had a high 
turnover rate among staff, often due to the low salaries that JTPA programs could pay. 
In these instances high turnover rates may cause some loss of program effectiveness 
because of the loss of experience, expertise, and institutional memory, although new 
staff contribute new ideas. In any case, it does appear that the effectiveness of the 401 
programs is directly proportional to the quality of the grantee staff. 

Section 401 Service Emphasis 

This section describes the design of INA 401 programs, what services are given 
emphasis and what services are minimized or omitted. We also comment on factors 
that influence decision-making effecting service provision throughout the sample. In 
order to comprehensively synthesize the overall picture, we first describe the general 
structures of INA 401 services in aggregate form to shed light on which services are 
emphasized in these programs. We then describe the services separately, noting causal 
factors in the differing sites related to priorities placed on service delivery. To the 
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extent that it is useful for analysis, we provide a contextual background for service 
selection and delivery. 

Data sources from which we drew to determine services and evaluate designs 
include: the 23 site visit reports conducted under this study, which contain descriptions 
of all services, computer-generated printouts of service activities of participants from 
the Annual Status Reports for all grantees in the sample for PY 96, and cost 
breakdowns for all grantees in the sample for PY 96 from the same source. 

Extent of Services 

Given the service options available to INA 401 grantees — namely Basic Skills 
Classroom Training, Occupational (or Job) Skills Classroom Training, On-The-Job 
Training, Work Experience, Community Service Employment, and Training 
Assistance— to what extent do INA grantees provide a variety of services/options for 
clients? PY 96 participant data indicate that six or 26% of our sampled grantees 
provide all six of the above listed educational and training services, twelve grantees (or 
52%) had participation in five of the six services, while 4 (or 17%) had participants in 
four services, and one grantee (or 4%) indicated participants in two services only. 
Combining the first two percentages, 78%, or over three-fourths, of the INA 401 
grantees provide at least five of the six allowable services. Figures indicate that most 
grantees make the effort to distribute funds across the spectrum of services to allow for 
multiple options for clients and multiple services. 

This coincides with the emphasis stated in most grantees' goals and objectives, 
namely that service is their main mission and purpose. That appears to translate into 
"as many services as possible for as many participants as possible". This does not 
mean, however, that all services are given equal importance nor that all services impact 
long-term goals for the participants, even if they are meant to be aligned with long- 
term goals and objectives of the INA 401 programs and grantee organizations. 

Comparison to Coals and Objectives 

We compared grantees' goals and objectives to the priorities and emphasis given 
to the service options— comparing what the programs purport to provide to what 
services they actually do implement and the extent to which participants are placed in 
the services. Three categories describe the findings: 

1 . Yes - Fifteen of the twenty-three INA 401 programs do appear to utilize 
service options that match their goals and objectives. For example, where terms such 
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as "long term" or "occupational skills" are used, the participation in classroom training 
is sizable; likewise when "job skills" and "reduce unemployment" are prominent, so 
are work site training and training assistance. These grantees go across the sample and 
include tribal, corporate, rural and urban grantees. 

2. Somewhat - Five of the twenty-three sites do a pretty effective job of matching 
services to goals and objectives. In these instances, grantees attempt to fulfill goals and 
objectives but fall short either because the goals are not attainable or lacking, because 
little training assistance is provided, because dropouts are not targeted as stated, 
because long term training is not emphasized or because higher percentages of funds 
are used in short-term activities than in long-term classroom training. These instances 
are found more frequently in rural, tribally-operated programs. Because they have 
been in operation since the CETA program, some have probably not adequately 
adjusted their goals and objectives over time to accurately reflect the funding cuts that 
have shifted priorities and reduced or eliminated some prior services . 

3. No - Only three of the twenty-three grantees provide services that are 
contradictory to their goals and objectives or do not provide the options they claim to. 
In two cases, objectives emphasized education, training and employment, but the 
grantees actually serve participants through Training Assistance only, Direct Placement 
referrals, weak basic skills services and spousal support services. These instances 
occur in rural tribally-funded programs with relatively small budgets, suggesting a need 
to reexamine goals and service priorities to better align and focus upon clients' needs 
and the most effective approaches to address those needs. 

Issues and Factors Influencing Service Designs 

What issues or factors determine how grantees design services and place 
priorities? Based upon an analysis of grantee documents and on-site interviews, we 
focus on seven categories that describe the major issues or factors that drive the design 
and delivery of services in the INA 401 sampled programs. 

1) The Labor Market - All programs are influenced by the labor market —by the 
ratio of available jobs to the number of unemployed. The majority of sites strongly 
indicate a need for more jobs/job development and a lack of economic development. 
Only two sites, one tribal and one organizational, indicated a sufficient supply of job 
opportunities, which afforded easier access for Direct Placements and referrals. Other 
labor market factors include the seasonal nature of the tourism industry at three sites 
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and the existence of casino employment as a positive factor at four sites (three tribal 
and one urban site) . One result of the lack of jobs is the reliance on Work Experience 
and Community Service Employment for placements in grantee offices and programs, 
both tribal and organizational. Seven grantees specifically noted tribal economic 
development and tribal job opportunities as a significant factor in their service 
selection. 

2) Reduction in INA 401 Funding - Thirteen grantees cited cuts in funding as a 
primary factor in service design and delivery. Citing cost effectiveness and low cost 
services as the rationale for service design and delivery, Training Assistance and Direct 
Placements are utilized more frequently. Conversely, funding cuts result in reduction 
in Classroom Training services, Support Services and Community Service 
Employment. Work Experience, in most programs, remains a priority, although fewer 
participants are placed than in prior years of higher funding. The basic skills 
Classroom Training which are operated/provided by grantees themselves rely on 
volunteers or underqualified instructors as one way to deal with inadequate budgets. 

3) External Problems with Private Sector and States - Twelve grantees indicated 
operational barriers as a result of communication or implementation problems with 
private sector employers and state agencies. The predominant issues center on 
employer problems in On-The-Job Training services. Among the stated problems with 
employers is their reluctance or refusal to hire participants at the conclusion of the 
training period, abusing the service, prejudice/racism, poor communications and failure 
to provide participant reports. Two tribal grantees stated that tribal sovereignty was not 
understood or recognized by the state agencies with which they coordinated. These 
external factors are very problematic and ones which grantees perceive to be out of 
their control to change. Only one grantee stated that "vigilant efforts on the part of the 
JTPA staff" brought about very successful coordination of the OJT service with the 
private sector in their service area. 

4) Geographic Size, Distance and Transportation - At least eight sites must meet 
challenges created by their large geographic size and resultant wide dispersion of the 
INA 401 service population. These sites are statewide, multi-state or multi-county and 
are primarily tribally operated, although a few are organizational grantees. Because the 
grantees cover both urban and rural areas, participants’ needs vary widely and these 
grantees usually attempt to provide a full range of services to meet those needs. 

Related to geographic size is the distance participants must travel to receive services. 
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Although satellite JTPA offices are set up to accommodate participants, the greatest 
issue appears to be transportation to attend Classroom Training services and work-site 
placements at some distance from their homes. Related to this issue is the 
understandable desire of participants to remain close to their families and communities. 

5) Existence of Other Programs/Services - Eleven programs specifically 
mentioned the existence of other social and educational services as a factor influencing 
their service priorities. At least nine grantees noted that social service needs such as 
child care, health and transportation were met by other funded programs, such as the 
tribal Indian Health Service and child care programs provided by the tribe or by JOBS, 
Head Start and other programs. Most significant were the references made by six 
grantees regarding the existence of basic skills educational programs funded by the 
Bureau of Indian Affairs (tribal programs) or by local community colleges and voc-tech 
schools. Where it is feasible and accessible, INA 401 grantees tend to make referrals 
to other programs for GED/ABE training. When funding decisions force reductions in 
services, the availability of these other services/programs influences reduction or 
elimination of basic skills classroom training in some INA 401 programs. However, 
since serving high school dropouts is a stated priority for many grantees, a concerted 
effort should be made to ensure that all participants are proficient in basic educational 
skills as a foundation for long-term employability. 

6) High Dropout Rates/Education Resources - Related to the above are the eight 
instances cited by grantees of the existence of high dropout rates among their targeted 
population as a factor determining their service design. In one instance, the public 
schools were described as ineffective for the Indian students who attended them; in 
another the high school had closed, leaving students with undeveloped basic educational 
skills or without diplomas. Certainly this is a serious factor given the current push for 
educational reform throughout the country. Perhaps INA 401 programs can join with 
tribal education departments, other federally funded programs with education 
components, states, public schools, community businesses, and other agencies to 
address this problem - to ensure the foundation is there for employability and a higher 
quality of life. Happily, one tribal grantee was able to cite a high graduation rate as a 
principal factor, allowing service priorities to focus on higher education, Occupational 
Skills Classroom Training, and employment placement. 

Related to educational training is the existence or lack of adequate service 
providers for Occupational Skills Classroom Training, as was made clear from the 
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results of the Administrator Survey presented in the previous chapter. Six site-visit 
grantees noted that this was a determining factor. While the majority could access 
provider campuses, in a few instances OSCRT was not a viable service priority because 
there were no facilities within a reasonable proximity to the service area. 

7) Miscellaneous Factors - This category includes factors mentioned only 
infrequently. One grantee stated that the Department of Labor's emphasis on long-term 
goals for participants guided its priorities and another stated that welfare reform was a 
factor. In reality, DOL's encouragement of long term solutions probably is an 
influence in more than one grantee's design and the full impact of welfare reform has 
yet to be realized in the INA 401 programs. 

Specific Service Emphasis 

To get a general picture of which services are emphasized, we can look at the 
participant levels in the various services and at the percentages of costs allocated to the 
services by the INA grantees. It is useful to look at these distributions in terms of 
percentages rather than absolute numbers, to get a better sense of service emphasis 
adjusted for program size. 

Which services are most emphasized in INA 401 programs? According to PY 96 
data, very clearly the Training Assistance and Work Experience services are the two, 
which INA 401 grantees utilize most frequently and under which the greatest 
percentages of participants are serviced. Training Assistance is a service provided by 
all grantees in our sample. Likewise, Work Experience is included in the services of 
all grantees sampled. Closely following these two top priorities is Occupational Skills 
Classroom Training, a service option included in all but one of our sampled twenty- 
three sites. Community Service Employment is the third most utilized option, with 
sixteen grantees indicating participation, and On-The-Job Training is the fourth with 
fourteen grantees exercising this option. Basic Skills Classroom Training is utilized by 
thirteen of our sampled sites, making it the least utilized. However, since over half of 
the INA 401 programs did indicate participants in this service, it is a significant part of 
the service design overall. The chart below indicates the range of percents of 
participants in each of the six education and training services: 
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Range of Percents of Participants PY 96 



Low % High % 



Basic Skills CRT 


3.2 


42 


Occupational Skills CRT 


.8 


70 


On-The-Job Training 


.3 


20 


Work Experience 


1.2 


69 


Community Service Employment 


1.5 


21 


Training Assistance 


1.2 


95 



Training Assistance. Noticeably, the range across all programs is widespread 
but we again see that the highest percentage of participants is found in Training 
Assistance, where one grantee serves 95% of its participants through this service. 
Understandably, this can mean many different things. Since some grantees include 
intake procedures, counseling and case management as well as special work readiness 
classes, testing and attendant supportive services under the umbrella of Training 
Assistance, this category can encompass many different things. In five sites over 50% 
participation is cited, and in eight sites the participant percentage is over 33%. 

These statistics simply underline that Training Assistance, a service which is 
usually one-on-one, which is usually short-term in nature, and which is more designed 
for short-term goal attainment, is one of the primary services of choice. Training 
Assistance is rarely confounded by transportation problems, as are some other 
employment and educational services, and, for the most part, does not have to rely on 
the cooperation or extensive coordination with other agencies. It is through Training 
Assistance that participants are generally introduced to the "world of work" or 
reeducated to the economic climate of the community, urban, rural or tribal. It is 
through Training Assistance that clients are helped to make transitions from one culture 
to another and to learn how to set their educational and occupational goals; it is also 
through this service that life and survival skills training surfaces— all basic to 
employability. 

To further broaden the concept, Supportive Services, in the form of shelter, 
meals, childcare, mileage, clothing, tools, medical, job search assistance, utilities, and 
stipends, frequently accompany Training Assistance, either in the form of financial 
assistance or referrals to other programs/agencies. It is easy to understand why T/A is 
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one of the most favored options. Do we conclude that employment then is the most 
important factor driving the design of INA 401 programs? Perhaps. 

Work Experience (WEX). This is another top priority for INA 401 programs, 
with two grantees indicating 59% of costs allocated to WEX and other PY 96 data 
showing that one grantee has 69% of its rather large participant base in WEX. Over 
half (13) of the sampled sites indicate that over Va of their participants are in WEX, 
with three grantees indicating more than Vz of their participant base is WEX. WEX is 
the first or second top option for half the sampled sites. 

Work Experience, as a service, is perceived by grantees as having several 
positive factors, which make it a priority service for the grantee, and for the clients. 
Among those are: 1) it does not require a long term commitment from either the 
participant or the employer, 2) placements can be made wherever there is a willing 
employer, either tribal, public, or not-for-profit, 3) placements are, more often than 
not, entry level, requiring less pre-service education or training, and participants are 
more easily placed, 4) participants sometimes recycle, entering and terminating WEX 
more than once or terminating WEX prior to entry into Community Service 
Employment or other services, and 5) it is a source of low-cost labor for the tribe, 
enabling tribes to provide services to their communities that they could not otherwise 
afford. Again, the utilization of WEX is another way to "provide as many services to 
as many as possible". While WEX is a "training" program it is just as much perceived 
as an "employment" program. Combine Work Experience participation with Training 
Assistance participation and yes, we are able to ascertain a priority given to 
employment - not in opposition to training or education but rather a greater emphasis or 
desire to place participants in a position to "earn a living" as quickly as their needs 
indicate. Sometimes this sacrifices a long-term goal for educational attainment, 
sometimes not. 

Occupational Skills Classroom Training (OSCRT). This is the third most 
utilized service based upon sampled grantees who opt to provide this service; twenty- 
one of the twenty- three study sites showed participation in OSCRT during PY '96. 
However the number of participants is somewhat lower than in WEX and TA services, 
with nine grantees placing up to one-fourth of their participants in this service. One 
grantee, a voc-tech institution, indicates 70% of participants in classroom training. At 
the low end, one grantee had. 8% of its participants enrolled in OSCRT. This 
emphasis is congruent with long term employability and skills enhancement and it is 
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heartening that grantees generally do place priority on OSCRT. Limited enrollment is 
said to be contained by the higher cost of OSCRT matriculation. Costs allocated 
ranged from 3% of INA 401 budgets upwards to 44% in PY '96. This cost data 
combines both OSCRT and Basic Skills CRT and is, therefore, not a discrete 
measurement. 

Community Service Employment (CSE). Community Service Employment is 
somewhat of a bridge between Work Experience and unsubsidized employment in a 
positive scenario. Placements in the not-for-profit sector seem to be one viable 
alternative to the problems cited with the private sector (or for-profit sector) . Along 
with Work Experience placements, it allows grantee organizations to place participants 
while also improving or adding to the work force of the tribal or organizational 
grantee, particularly the tribal grantees. One grantee sees its programs as another way 
of "improving the capability of the nation" . CSE is the service that is fourth among 
priorities, being offered in sixteen of the twenty-three grantee sites. 

Conversely, fewer participants are placed in CSE overall than in TA, WEX or 
OSCRT. Where the service is provided, less than one-fourth of participants are placed 
in CSE throughout the sampled sites. Participant data in PY '96 indicates a low end of 
percent of participants at 1.5% in an urban setting and a high end of 21% of 
participants in a large multi-state tribal setting. CSE is one of the services eliminated 
or lessened when budget cuts occur, because the salary costs are somewhat greater and 
the job skills required tend to be more stringent than WEX. Another factor is the 
budget cap placed on CSE, making it a less flexible option. Costs allocated to this 
service range from .5% at a voc-ed facility to 60% of the budget of a rural tribal 
grantee. 

On-The-Job Training (OJT). This is the fifth service priority based upon 
participation data for PY 96. Fourteen sampled grantees provided this training and 
employment service. There are fewer participants overall than in any other service, 
however, and the lowest range of participation. The lowest percent of participation is 
.3% in an urban organizational program with the highest percentage of participants 
reported at 20% in a statewide combined urban and rural tribal program. According to 
interviewees, there are multiple problems with OJT providers in the private sector, 
which have been previously discussed. It appears as though service providers are 
hesitant to commit to hiring participants in full-time positions after the subsidized 
training period has ended. The other prevalent factor is the shortage of employment 
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opportunities in underdeveloped regions, particularly in isolated, rural service areas. 
Because of the multiple confounding factors, OJT is another of the services that are 
eliminated or cut back when INA 401 funds diminish. PY 96 cost data indicate lower 
percents of costs are allocated for this service, with the low end of .3% of funds 
allocated at a large multi-state rural tribal program to the high end of 24% of funds at 
an organizational state-wide program. 

Basic Skills Classroom Training (BSCRT). Lowest in service priorities, based 
on PY 96 participant data, is training for basic skills, including GED and ABE 
training. Thirteen of the twenty- three sites provide this service through a variety of 
service providers: community colleges, voc-tech schools, high schools, alternative 
schools and grantee on-site classrooms. This service is one where the quality of 
services fluctuates based on funding levels, on the qualifications of instructors, on 
teaching methods and materials, and on other services or lack of them. While some 
programs require participants to have diplomas/GEDs or score at age/grade reading and 
math levels before enrollment, it is not a uniform practice. The existence of other 
sources of funds and other basic skills programs allows some grantees to refer 
participants out rather than provide the service through the INA 401 program. 
Coincidentally, many participants who are dropouts or lack basic skills constitute a 
significant portion of the JTPA client base. Conversely only two grantees report 
significant participation percentages in this activity, of 25% - 33%. Along with OJT 
and CSE, percentages of BSCRT participants overall are lower than TA, WEX, and 
OSCRT, with the low range at 3.2% participating in an urban program and 42% 
participating in an isolated rural tribal program. 

Summary 

In summary there are multiple external and internal factors which determine how 
INA 401 grantees design and deliver services. No two sites are identical or have 
identical priorities. As economies, labor markets, funding, organizational and tribal 
priorities, and clients needs and goals differ, so do the services. 
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V. SERVICES IN SUPPORT OF TRAINING 



This chapter describes activities and services related to selecting clients and 
supporting them during training. These activities include targeting, outreach and 
recruitment, intake and assessment, and case management and supportive services. Our 
goal for this chapter is to describe strategies used by case study grantees to support 
participants while in training and examine whether the design and implementation of 
these activities are consistent with the model of high-quality training. 

As discussed in earlier chapters, the Indian and Native American population 
eligible for the Section 401 program is tremendously diverse, as are their needs for 
services. Our model of responsive services depicted in Chapter I suggests that 
activities in support of training should be tailored to the circumstances of the local 
context and support the unique needs of participants and communities. According to 
this model, program quality can be enhanced if grantees: (1) devise and implement 
strategies to reach the target population, and (2) review the service needs among 
eligible participants and assess their own program’s and other agencies’ service 
capabilities to meet those needs. Such efforts can ensure that programs develop service 
strategies that take into account their clients’ special circumstances while also avoiding 
a duplication of services. 

Below we discuss strategies and practices that case study grantees implemented to 
support the development of high-quality, responsive activities and services that directly 
impact decisions about service delivery. We also highlight the challenges grantees face 
in promoting activities intended to support participants, and the strategies developed to 
address those challenges. 

Targeting Participants 

Because the need for Section 401 services far outweighs the grantees’ capacity, 
some grantees identify specific target groups for employment and training services. 
Identifying a priority group for services can be important because most grantees, due to 
their limited program budgets, can often use their resources most efficiently and 
effectively when focusing attention on certain types of clients over others. Moreover, 
developing targeting goals and effective outreach mechanisms can promote equitable 
access to services for all members of the Indian and Native American community, 
particular those living in remote areas. 
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One of the first decisions some grantees make, therefore, is to determine who 
among the eligible applicants is enrolled for services. These decisions are often 
difficult. In every program, grantee staff reported that their eligible population consists 
of youth and adults, high school graduates and dropouts, and those with some or no 
previous work experience. When asked about targeting decisions, grantee staff insisted 
that all eligible Indians and Native Americans in their local communities were by 
definition disadvantaged and in need of services. 1 Grantees therefore tended to recruit 
widely to meet the needs of any potential INA participant who sought services. 
Moreover, there is a strong ethic among Native American social service agencies that 
no one should be turned away who needs services. 

Despite these facts, some grantees identified several segments of the eligible 
population that they were particularly interested in reaching and for whom they felt 
JTPA services were most appropriate. Decisions about which groups to target were 
generally made because grantees strongly believed that JTPA services would have the 
most impact when offered to certain types of participants over others, and they believed 
that program resources were better suited to meet some needs over others. 

Based on participant file reviews and discussions with program staff and 
participants, we learned that, of those case study grantees who mentioned targeting, two 
key groups for employment and training services were identified: (1) the hard-to-serve, 
and (2) participants with specific characteristics. 

Targeting Hard-to-Serve Participants 

Some grantees explicitly targeted hard-to-serve groups because they believed that 
this subset of the population was most in need of services and a group for whom 
services could have the most impact. Grantees identified persons deemed hard-to-serve 
as those with limited employment skills or education, or who are basic skills deficient, 
homeless, or at-risk youth. Others believed that all members of the eligible population 
were hard-to-serve and they thus had difficulty serving one group over another. To 
address this challenge, some grantees identified participants considered "most in need” 
by developing specific strategies for recruitment or enrollment. Some, for example, 
established a “no repeaters” policy that would give preference to eligible participants 



1 According to JTPA Section 401 regulations, eligible participants include those who are Indian, Native 
Alaskan, or Native Hawaiian as determined by the grantee, and who are economically disadvantaged or 
unemployed or underemployed. 
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seeking JTPA services for the first time. Other grantees identified the most in need by 
soliciting referrals from human service agencies that serve hard-to-serve individuals, 
including, for example, homeless shelters, foster homes, and agencies providing mental 
health services for persons facing life challenges. Because of the challenge of 
identifying a hard-to-serve group, others resorted to enrolling individuals on a “first 
come, first served” basis and served as many people as program resources would 
allow. 

Among grantees that identified hard-to-serve target groups, there was a moderate 
disparity between their targeting goals and the types of participants served. For 
example, several grantees, because they wanted to ensure a positive termination, served 
a large number of participants who already had some college education and a solid 
employment history. Although some of these participants may have required a skills 
upgrade to be competitive in the labor force, they were sometimes served first before 
participants identified as hard-to-serve. 

Targeting Participants with Specific Characteristics 

Some grantees targeted participants with specific characteristics, including 
possession of occupational skills that were in demand in the local economy, youth 
participants, and motivated participants. The decision to target these groups was made 
because grantees believed that these groups would most benefit from specific service 
activities over other groups. The grantees’ decision was strongly influenced by several 
factors, including limited program resources and employer demands, particularly in 
local sites that emphasized the provision of subsidized employment activities. 

Specific circumstances that led grantees to identify target groups varied widely. 
Several grantees designed their programs with an emphasis on specific service 
activities, such as Work Experience or classroom training, and operated under the 
premise that they would serve certain groups that could enroll in these activities. Some 
grantees enrolled participants with limited employment skills and work experience so 
that they could participate in subsidized opportunities designed for entry-level workers. 
In another case, a grantee that emphasizes Work Experience in a day care center targets 
primarily women caregivers. Similarly, grantees that offer primarily classroom 
training activities target either high school dropouts or high school graduates who are 
interested in post-secondary education. 
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Several grantees reported that they explicitly target “motivated” participants 
whom they felt would succeed in the JTPA program. For example, one grantee who 
faced high attrition in its service activities in the past tried to identify participants who 
appeared motivated. Accordingly, it eschews on-the-spot enrollments and instead 
developed a system to send reminder notices to interested applicants to attend an 
orientation session. Applicants who fail to show up after two notices are assumed to be 
poorly motivated and would not be enrolled. This grantee felt that this newly devised 
system would help address the problem it was having with high attrition. 

In general, grantees that established specific targeting goals strongly believe that 
identifying a priority group was beneficial for the participants and the worksites or 
service providers, because then the needs of both would be met. 

Outreach and Recruiting Strategies 

Our model of responsive services suggests that programs can be effective in 
ensuring equitable access by developing outreach and recruitment procedures consistent 
with targeting goals. However, nearly all grantees reported that, because of the high 
demand for JTPA services, they did not need to aggressively recruit to fill available 
slots. Participant outreach and recruitment was therefore not given a high priority in a 
large number of case study grantees. Instead, our sampled grantees relied heavily on 
indirect recruiting practices to make clients aware of Section 401 services, although 
some used more direct, proactive methods. We describe these methods below. 

Indirect Recruiting 

The first approach— indirect recruiting— constitutes the primary method cited by 
sampled grantees. This approach requires minimal effort by the grantees to reach 
potential participants and can be viewed as a “passive” recruitment strategy. Indirect 
recruiting typically consists of the following: 

• Word of mouth referrals, or the use of the “moccasin telegraph” 
whereby current or former Section 401 participants notify their friends 
or relatives about services and encourage them to enroll. 

• Referrals from other community agencies, including community 
colleges, tribal agencies, social service agencies, and other employment 
and training systems, such as Employment Service. 

• Distribution of program information to community agencies to 
encourage referrals and increase the community’s awareness of services 
provided through Section 401 programs. 
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Word-of-mouth referrals. The most prominent method of drawing clients to 
JTPA services was through word-of-mouth referrals. Most grantees, especially tribal 
grantees, reported they use this method as the primary recruitment mechanism and felt 
that this method, over others, was very effective because of the close-knit relationships 
established within IN A communities. Grantees reported that their long history of 
providing services, along with the reputation they had established, further facilitated 
word-of-mouth referrals. The vast majority of participants interviewed also indicated 
that word-of-mouth referrals were the primary means by which they became aware of 
Section 401 services. 

The advantage to relying primarily on this recruiting method is that it requires 
little or no supplemental financial or staff effort to reach participants and it almost 
always yields an adequate flow of clients seeking services. The main disadvantage to 
this recruiting approach, however, is that some potential enrollees who are desirous of 
services but are not well connected to the community’s social network may be unaware 
of the program and hence will be essentially precluded from participating. For 
example, several participants mentioned to us during our site visits that they had 
learned about the program from a friend or relative and found it to be very helpful, but 
wished they had known about it years before; they wondered why the program didn’t 
advertise more. Similarly, participants living in remote areas may also be disconnected 
from the service stream. 

Referrals from agencies. Another indirect recruiting method practiced by 
sampled grantees is to solicit participant referrals from local community agencies and 
employers. Half of our case study grantees generated referrals by using their 
relationships with employers, schools and colleges, and a variety of tribal and non- 
tribal agencies providing social, educational, health, and family services. Referrals 
from these agencies are viewed as an indirect recruiting practice because relationships 
with them can lead to a steady client flow with little supplemental effort by grantees. 

Referral linkages with agencies are often established informally— grantee staff 
typically rely on personal relationships between tribal and non-tribal agencies to 
develop and maintain linkages. Some case study grantees developed mechanisms to 
sustain institutional linkages by holding formal weekly or quarterly meetings with other 
community-based organizations and service providers to promote informational 
exchange and cross-agency collaboration. One grantee with limited funds sought 
creative ways to extend its recruitment activities by getting involved in a variety of 
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community groups and committees to inform the staff of other programs and agencies 
about the JTPA program. 

A number of grantees also developed linkages with employers to facilitate 
participant referrals. Strong working relationships with employers enabled grantees to 
identify job openings for training in subsidized employment and increased referrals 
from employers. Although linkages with employers proved valuable in the long run to 
grantees and participants, our respondents admitted that participants recruited through 
this mechanism often could have found employment without the assistance of JTPA. 

Distribution of program materials. Grantees also developed marketing 
strategies to reach a wide audience and made a concerted effort to distribute 
information about JTPA services to a wide range of agencies and service providers, 
including employers, social and health service providers, and training providers. 
Grantees typically placed brochures, flyers, and posters in schools, community 
colleges, and social service agencies. 

Direct Recruiting 

Some grantees also practiced direct recruiting strategies by using “active” 
methods— those that require substantial resources, including staff time and program 
funds to reach potential participants. Examples of direct recruiting activities practiced 
by case study grantees included: (1) advertisements through the local media, (2) 
presentations to key staff at community-based organizations, (3) establishing satellite 
offices dispersed throughout the service area, and (4) sponsoring community-wide 
activities to promote JTPA services. 

Advertisements through the media. Many grantees, particularly large grantees 
operating in large service areas, recruit participants by advertising through the local 
media. This recruitment method is considered active because it requires the 
commitment of extra resources. Most grantees advertise in Indian newspapers and 
newsletters, while some grantees went further by making appearances on television and 
radio talk shows, and making public service announcements on the radio and television. 
These ads encouraged participants to call or visit the JTPA offices for more 
information. Advertising in the local media was generally used sparingly, however, 
because of the high cost of advertising. 
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Other advertising activities included developing videos and accompanying 
publications to distribute to participants and employers in local agencies located 
throughout the IN A communities. 

Staff presentations. A large number of grantees practiced direct recruitment by 
making presentations to staff in local agencies. Grantee staff conducted presentations at 
community meetings, job fairs, community-based organizations, and schools to 
disseminate program materials and publicize JTPA services. 

Satellite offices. At least seven grantees in our sample, who served large and 
geographically diverse areas, maintain field offices to conduct outreach to potential 
participants. Maintaining field offices, case study grantees find, supports their efforts 
to recruit a broad spectrum of participants while ensuring that members of the Indian 
and Native American community have convenient access to services. For example, one 
grantee outstations its staff in a low-income area inside a housing complex once a 
month to facilitate participant recruitment. 

A number of grantees established other innovative strategies to inform the 
community about JTPA services. In several rural and urban sites, grantees outstation 
staff in colleges and in other network agencies on a weekly basis to reach potential 
participants. Some grantees conduct intake activities away from the JTPA office to 
make it easier for participants to enroll for services. Staff from several grantees visit 
the homes of clients to promote program services. Another grantee serving 41 counties 
located in rural, suburban, and urban communities requires staff to travel throughout 
the service areas to reach clients. Some grantees also try to make it convenient for 
clients by accepting collect calls from potential participants inquiring about services. 

Community-wide activities. Many case study grantees promote JTPA services 
by sponsoring community-wide events such as POW-WOWs and job fairs that attract a 
large number of individuals from the community. One grantee sponsors job fairs twice 
a year to recruit participants and to make employers aware of JTPA services. 

Limitations Influencing Participant Recruitment 

Several important factors limited the grantees’ ability to effectively recruit and 
reach potential participants. These factors include the following: 

• Limited program and human resources affected grantees’ ability to 
conduct extensive recruiting activities. Due to limited budgets, most 
grantees are unable to devote very many resources to recruitment. 
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Because most grantees already have more than ample case loads, more 
extensive recruitment is in any case felt to be unnecessary and a waste 
of resources. 

• The large service area, particularly for tribal grantees, inhibited 
grantees from conducting extensive participant recruitment. Although 
some grantees expended resources to establish field offices and advertise 
through the local media to reach participants, grantees with limited 
program funds were unable to effectively recruit participants dispersed 
throughout the service area. By implication, some potential clients are 
unintentionally denied access to services. 

In short, grantees used a variety of outreach and recruitment methods to reach 
potential participants. These methods generally appear to be very effective for their 
particular areas, but some strategies had certain disadvantages. The challenge, reported 
a number of grantees, was to develop appropriate outreach strategies that could 
effectively reach a large number of clients given the limited resources available. 

Intake and Assessment 

After participants are recruited for services, they undergo the intake and 
assessment process, which is an initial step towards. the development of individual 
service plans. Consistent with the model of responsive services detailed in Chapter I, 
grantees should develop procedures for intake and assessment activities that will 
identify participants’ strengths and weaknesses in order to inform the service planning 
process. As described below, intake and assessment services vary significantly across 
case study grantees and, for the most part, these activities are practiced informally. 

Intake Activities 

Applicants seeking services through Section 401 programs undergo an intake 
process, which typically includes (1) an orientation to services, (2) eligibility 
determination, and (3) the completion of an intake application. Many grantees integrate 
these activities to obtain client information and to make efficient use of applicants’ time 
spent at the JTPA office. 

Orientation activities typically consist of an overview of services and eligibility 
requirements. Five case study grantees offer a formal group orientation to services. 

One grantee views the orientation process as an integral part of career planning and 
provides a six-hour orientation session that includes eligibility determination, self 
assessment, and an overview of community resources through presentations by social 
service representatives from the local community. 
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As part of the intake process, grantees developed intake applications that solicit 
basic demographic data, educational and employment background, employment and 
educational goals, and supportive service needs. Two case study sites providing an 
array of educational and social services developed a “common intake form” to simplify 
access to multiple services offered by the grantee and the JTPA program enrollment 
process. 

As part of the intake process, participants meet with the intake counselor or case 
manager, who collects the necessary information and paperwork from participants to 
determine eligibility for services. Respondents reported difficult challenges in 
collecting eligibility documents from potential participants and expressed concern over 
the burdensome nature of collecting documents and filling out forms. Respondents 
reported that many applicants were turned away for services because they were unable 
to demonstrate their Indian ancestry. 

Once the intake was completed, information collected from the intake application 
functioned as a starting point for the assessment of services. 

Assessment Activities 

The next step to understanding the service needs of clients is to conduct an 
assessment of the clients’ skills, interests, and service needs. Section 401 programs are 
exempt from the statutory requirements in JTPA’s Title II to conduct an “objective 
assessment” of all clients, and grantees thus have ample discretion in designing 
assessment activities to meet the particular needs of the Indian and Native American 
community. 

A key feature of our model of responsive services suggests that assessment of 
participants’ skills, interests, and needs should be tailored to the participants’ unique 
backgrounds and circumstances. Understanding the service needs of participants is 
especially critical in Section 401 programs because individuals seeking services present 
diverse needs and career interests. According to respondents, many participants suffer 
from low self-esteem, substance abuse, lack of employment skills, lack of knowledge 
about the world of work, and other life challenges. Identifying these barriers through 
assessment is critical. Similarly, clients seek services often with well-defined 
objectives in mind, from undergoing long-term post-secondary education to receiving 
an immediate job referral. Grantees must therefore adapt the assessment process in 
ways that are useful and practical for the clients. Below we discuss the areas and 
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methods of assessment and the arrangement grantees make for conducting assessment 
activities. 

Areas and Methods of Assessment 

All Section 401 grantees conduct some form of assessment of participants’ service 
needs and assess for a variety of personal characteristics, including the following: 

• Basic skills proficiency 

• Aptitudes 

• Interests 

• Supportive service needs 

Nationally, data from the grantee survey, as presented in Chapter III, revealed 
that most grantees conduct assessment in a variety of areas for all or most clients— 76% 
of grantees assess for reading skills, 60% assess for math skills, and 69% assess for life 
skills. Nearly all survey respondents assess for barriers to participation in training or 
employment (97%) and previous work experience (96%) , while 85% assess for 
occupational interests. 

Assessing for basic skills (reading and math) proficiency occurs in the majority of 
the case study grantees as well. Sampled case study grantees reported that basic skills 
assessment is provided either on-site or through a provider, most commonly using the 
Test of Adult Basic Education (TABE), the Comprehensive Adult Student Assessment 
System (CASAS), or the Job Corps test. 2 . However, not all clients are necessarily 
tested. For example, many grantees provide the option of not testing for basic skills if 
participants are currently attending or have attended college or if they are interested in 
just receiving direct placement assistance. 

Many grantees assess participants’ aptitudes in a variety of occupational skills, 
including, for example, motor skills, spatial reasoning, and typing speed using formal, 
commercially-developed tests or self reports. Assessing for proficiency in specific 
aptitudes was typically reserved for subsidized employment activities to better 
understand participants’ skill levels. 



2 One grantee from our sample reported that some commercially developed basic skills tests are 
culturally biased against Indians and Native Americans and its staff are therefore seeking alternative 
methods to test for basic skill levels. 
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Understanding participants’ occupational interests is another goal of assessment 
reported by some grantees. About half of case study sites conducted a career interest 
inventory to gauge participants’ occupational and career interests and used formal paper 
and pencil tests such as the Career Occupational Preference System (COPS) , self- 
reports, the intake application, and informal interviews with grantee staff. A number of 
case study grantees also measure participants’ values in an effort to understand the 
“whole” person. 

The majority of grantees directly provide some form of supportive services and 
almost all of case study grantees assess for supportive service needs. All grantees 
emphasized that many participants face substantial barriers to employment, (e.g. lack of 
housing, lack of childcare) that must be addressed to be successful in the Section 401 
JTPA program. As such, understanding the scope of participants’ supportive service 
needs informed the development of practical and appropriate service planning strategies 
to support participants during training. 

Grantees assess for several types of supportive service needs, including: 

• Sustenance needs such as transportation, housing, nutritional and other 
training- related needs. 

• Logistical needs, such as arrangements for childcare and transportation 
to attend training. 

• Personal needs, such as health related needs, substance abuse 
counseling, and other emotional needs. 

Nearly all grantees assessing for supportive service needs use informal methods, 
including informal interviews, self-reported information on the intake application, and 
observations conducted by grantee staff. In one case however, the grantee developed a 
formal procedure to assess supportive service needs by using an internally developed 
form called a “Financial Needs Analysis Form.” This one-page document lists expense 
categories and participants’ total monthly costs on one side and their total monthly 
resources on another side. This information helps the participants and grantee 
understand the participants’ financial situation and enables grantee staff to determine 
which participants are in most need of supportive services. 

Overall, we found that the majority of grantees practiced responsive assessment 
services by tailoring assessment activities appropriately, depending on the clients’ 
interests and employment goals. For example, a number of grantees provide 
assessment testing to individuals with specific characteristics, including those lacking a 
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high school diploma or those whose interest in specific service activities require 
assessment testing in reading and/or math. One grantee paid particular attention to the 
experiences of its participants by not testing “older" applicants who “would be 
offended” if they had to be tested in order to enroll in specific services. 

The challenge that most sampled grantees face, however, is effectively 
interpreting assessment results to inform participants’ service planning strategy. 

Indeed, even grantees with well-developed assessment procedures face difficulty linking 
assessment results to participants’ service strategies, as will be discussed shortly. 

Arrangements for Assessment Activities 

Assessment services are provided by either grantees or local service providers. 
Five out of 23 case study grantees, or 22%, contract assessment services to community- 
based entities such as community colleges and employment and training systems such as 
Job Service. Assessment providers typically assess for basic skills proficiency and 
career interests using a standard battery of paper and pencil tests. The remainder of the 
sampled case study grantees conducted assessment in-house. 

The majority of case study sites thus conduct their own assessment activities. 
Typically, intake staff and case managers document participants’ educational and 
employment barriers, occupational and training interests, career goals, and supportive 
service needs using commercially developed paper pencil tests and customized intake 
applications, informal interviews, and observations by grantee staff. One grantee 
developed an extensive, 14-page intake application to solicit information about 
participants’ work history, and educational and career goals, in addition to information 
on participants’ financial and family situations. 

As noted, some grantees make arrangements with outside providers to conduct 
assessment. This method reportedly made a difference in decreasing the grantee staff’s 
workload and helped solidify linkages with community agencies. This arrangement 
also addressed the lack of human resource capacity that some grantees faced to 
effectively conduct assessment. To avoid a duplication of services, some grantees 
divided up the assessment responsibility. For example, some grantees assess for 
occupational interests, while the service providers assess for basic skills and aptitudes. 
In some cases, additional assessment of participants’ educational levels and work 
maturity are conducted at the service provider level so they can tailor instruction and 
training to participants’ specific needs. 
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Service Planning 

After assessment, participants meet with a counselor to begin the process of 
service planning— a process in which participants and grantee staff should be jointly 
involved in establishing participants’ short-term and long-term career goals. As 
emphasized in the model of responsive services, the individualized service planning 
process should be carefully crafted to ensure that applicants receive services that are 
most appropriate to their individual circumstances as determined by assessment results 
and that goals are jointly established between the staff and the applicants. In general, 
when making decisions about service planning, grantees adhered to these principles. 
Additional factors that were considered included the availability of services and service 
providers, and in many cases, transportation challenges and labor market options. 

All case study grantees developed a formal document to facilitate and document 
service planning— commonly called the Employment Development Plan (EDP)— a 
document that describes participants’ barriers to training and employment, educational 
and career goals, and service activities to support those goals. Similar to other DOL- 
funded employment and training programs, the EDP is intended to serve as a strategic 
service-planning document that records participants’ progress, and their service needs, 
and should be updated as participants move through the program. 

Although all case study grantees developed an EDP as a way to initiate the 
service planning process, the amount of guidance received during service planning 
varied considerably. We learned that, while most grantee staff were careful about 
matching participants to service activities consistent with their interests, few programs 
established direct procedures to help clients decide on a career path or direction through 
the use of assessment results. Program staff reported that, often, clients visit JTPA 
offices with a clear idea of a service activity, most commonly, job placement or a 
specific training program in mind. Several programs reported that clients often want to 
enroll in subsidized employment opportunities only, even though grantee staff 
acknowledged that their assessment results and employment needs suggest a different 
activity. 

Grantees were often constrained by several factors in developing responsive 
service planning strategies, including participants’ interests in employment only and 
their immediate need for income. Because of these constraints, many grantees often 
failed to develop service plans that addressed long-term employment needs of 
participants and, instead, opted for immediate solutions to “help out” participants, 




V-13 



142 



which often involved placement in subsidized employment for varying periods of time. 
During site visits, we observed that participants were assigned to subsidized 
employment opportunities even when they tested to be basic skills deficient. 

Despite challenges that many sampled grantees face, we discovered several 
examples in which the EDP and service planning process was inventive and well 
developed. These examples are presented below. 

Some case study grantees made service planning decisions based on individual life 
experiences, including participants’ educational and employment history. Eight 
grantees considered participants’ basic skills test results when developing the 
service plan and referred participants to basic skills and GED service providers 
based on the need identified in assessment results. Although a number of 
participants did not attend basic skills training, the grantee staff encouraged 
participants to attend. 

One grantee uses several staff members to conduct individual assessment. Each 
staff member assesses for a variety of needs, including financial and supportive 
service needs, health needs, and other personal needs. Staff members share 
information on each potential participant and make recommendations on “next 
steps” to achieve service goals. Typically, counselors make note of health needs, 
such as counseling for substance abuse, and refer participants to an in-house 
substance abuse counselor prior to receiving services. 

The service planning process in three case study sites was facilitated by 
participants, many of whom were already enrolled as students in a local college 
or university and sought JTPA services to support them during training. These 
participants already knew what services they wanted and program staff reviewed 
their profiles and assigned them to activities that interested them, after the staff 
assessed their needs. 

Factors Influencing Service Planning 

Discussions with a number of respondents and a review of participant files 
revealed that several factors influence the development of individualized service plans 
and the match to services. Consistent with the results from the Administrator Survey 
(see Table III-12 in Chapter III), these factors include (1) geographic challenges, (2) the 
availability of service providers, (3) participants’ interests, (4) assessment results, (5) 
funding constraints, and (6) statutory restrictions on the use of funds. 

• Transportation challenges. In the majority of our case study grantees, 
transportation concerns represented a key factor in providing the 
services outlined in the individual service plan. Participants are 
sometimes assigned to services in their residential vicinity and matched 
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to worksites or other service providers that are closest to their homes, 
even if the service activities are not fully consistent with their longer- 
term career goals. 

• Availability of service providers. The availability of service providers is 
another important influence on deciding the types of services 
participants will receive. In some areas, limited service providers 
meant that participants are not offered a menu of services, but only the 
narrower range of services that are available. 

• Participants’ interests. Although participants’ interests were not always 
the driving force (see the results from the Administrator Survey) , the 
individual service strategy was heavily influenced by the participants’ 
own expressed preferences. 

• Assessment results. For many grantees, assessment results are used to 
identify employment barriers, and grantee staff view assessment results 
as important to the development of the service plan. These grantees 
typically rely on results from basic skills tests to determine participants’ 
skill levels and subsequently make referrals to basic skills service 
providers as needed. Some case study grantees even establish a policy 
stipulating that participants deemed basic skills deficient must attend 
basic skills remediation or GED classes prior to receiving additional 
services. The majority of case study grantees, however, do not rely 
primarily on assessment results when developing the service plan. 

• Funding constraints. Under this heading we include both the grantees’ 
own budgets, which limit the types of services that can be provided and 
the duration of training, and the participants’ budget constraints, which 
often make it difficult for them to undergo long-term training in the 
absence of a steady source of income. 

• Restrictions on the use of funds. Although JTPA allows wide discretion 
on the types of services that can be provided, there are some statutory 
or regulatory restrictions on how funds can be used, such as restrictions 
on the length of OJT. 

Overall, we found only a few examples in which service planning was responsive 
to the long-term training needs of participants. Several factors and concerns, most 
notably participants’ stated interest in gaining immediate employment and limited 
service options, impacted the effectiveness service planning. 

Case Management Services 

Once individual service plans are developed, they need to be effectively carried 
out through case management. Case managers track and support participants while they 
progress through their planned service activities and can function as advocates for 



V-15 



144 



participants during training and help them address their skill deficits by brokering for 
resources and supportive services within the community. Below we discuss how the 23 
case study grantees practiced case management and highlight the staffing arrangements 
for case management, the frequency of contact between case managers and participants, 
and the level of intensity of case management activities. 

Staffing Arrangements for Case Management 

Providing support and advocacy for participants during training is a critical 
component to our model of responsive services. The 23 sampled grantees varied in 
their approaches to staffing case managers. In the majority of cases, individual case 
managers are assigned to manage a caseload of participants, while a small portion of 
grantees practiced a team approach to managing participants. The majority of sampled 
grantees reported that staffing shortages greatly influenced staffing patterns. 

In five case study sites, several staff members — consisting of intake staff and job 
developers — work together and share client information to help monitor participants’ 
progress and support them in overcoming difficulties encountered while in training. In 
two of these grantees, the entire JTPA staff with overlapping roles in intake and job 
development all function as case managers and provide ongoing assistance and 
intervention to help participants tap into community resources and address their 
personal needs (e.g. housing, child care). In two other programs, the JTPA and service 
provider staff jointly, including employers and instructors, oversee the participants’ 
progress and help each other identify potential barriers participants may face during 
training. For example, in one site, an employer frequently visits grantee staff to 
provide updates on participants’ progress, while grantee staff follow up on the 
employers’ feedback by contacting the participants directly to gather information about 
their needs. The remaining grantee of these five conducts regularly scheduled 
“tracking” meetings with a team of case managers to solicit information about client 
progress and follow up on referrals made to agencies providing services. Two grantees 
switched from a one-on-one case management approach to a team approach to reduce 
the level of “hand holding” to encourage participants to be self-sufficient. 

The remaining 18 sampled grantees assigned one staff member to provide one-on- 
one case management to a caseload of participants. These grantees typically start case 
management activities at intake and continue to monitor and track participants as they 
progress through training. 
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Intensity of Case Management 

In general, the intensity of case management activities varied significantly. Case 
study grantee staff indicated that because they enroll a large percentage of “hard to- 
serve” participants, case management services need to be highly individualized and the 
level of intensity should be appropriate for each participants’ life circumstances and 
service activities. A strategy that most grantees commonly used is to vary the 
frequency and intensity of case management activities on the basis of: 

• The types of participants served. Some grantees target intensive case 
management services to persons experiencing severe hardships, 
including those who are substance abusers, former offenders, or in need 
of medical attention, and they meet less frequently with participants who 
are progressing well. According to respondents, this strategy ensures 
that case management services are tailored to the needs of individual 
participants and are used with the greatest efficiency. 

• The types of services provided. A large majority of grantees tailor the 
intensity of case management to the activities in which participants are 
enrolled. For example, some grantee staff visit participants enrolled in 
work experience more often than those enrolled in occupational skills 
training. Conversely, one site provides more intensive case 
management to participants enrolled in occupational skills training than 
those enrolled in work experience. Several other grantees provide least 
intensive case management to participants seeking financial support 
while enrolled in a college program. Targeting case management 
services can promote the efficient use of staff resources; however, we 
suggest grantees use this approach with caution, as many participants 
may inadvertently not receive adequate case management when needed. 

• The stage of program enrollment. Several grantees provide intensive 
case management at the beginning of program enrollment or towards the 
end of the program, depending on the type of service activities in which 
participants are enrolled. For example, two grantees provide intensive 
case management early in program enrollment until grantee staff 
establish a good rapport with the participants and assure themselves that 
the client is progressing well. 

In the majority of grantees, the types of case management activities included 
administrative tasks such as reviewing and collecting timesheets for subsidized 
employment, monitoring attendance records, and distributing student checks for work- 
related training. Although most grantee staff were expected to visit participants in their 
classrooms and worksites weekly or monthly, a number of grantees do not enforce this 
policy. However, some case managers informally “check on” participants “to see how 
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they were doing” and, when necessary, seek additional resources to address problems 
that arise. 

We found many examples in which case managers were caring and dedicated and 
instances in which case management activities were carefully planned and well 
executed. Some examples are highlighted below. 

At one grantee, case management services are considered a key component to 
service delivery and grantee staff often go beyond the call of duty to provide 
client support. The case managers spend the bulk of their time gathering 
information about different programs and funding streams that would enable 
clients to take advantage of other opportunities. One participant had to move in 
with relatives in order to avoid what would have become a formidable 
transportation challenge. However, the house was already very crowded. The 
case manager found a housing option through the end of training “just in case” 
the participant would have to move. 

This same grantee is able to work jointly with staff at the service provider level to 
provide responsive case management. A case manager notified the instructor of a 
participant facing temporary child care barriers and the instructor allowed the 
participant to bring one of his children to class. Case managers also work closely 
with counselors at the service-provider level to identify tutoring opportunities for 
participants. 

While some grantees developed careful strategies in providing responsive case 
management, we encountered instances in which case management was not an explicit 
priority in service delivery. This was due in part to high caseloads and the large 
service area for which case managers are responsible. As a result, case managers only 
had enough time to check in with clients periodically, through telephone conversations 
or visits to worksites and classrooms. For example, in one program, a single case 
manager visits all worksites being used for work experience in one day once a quarter. 

We also found evidence that case management activities are practiced informally 
and often not recorded in participants’ individualized service plans. During our review 
of participant files, we saw little evidence of client contact and follow-up but 
discussions with case managers reveal that they do in fact provide ongoing support 
when needed. For example, a case manager indicated that she paid special attention to 
a participant at risk of dropping out of the program and called him regularly in the 
morning to remind him to go to his worksite, but no evidence of this was in the case 
files. 
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To summarize, the intensity and quality of case management depended to a large 
extent on the participants’ unique circumstances. We found that frequent, intense, and 
personal contact between case managers and participants appear to be instrumental to a 
successful case management model. Unfortunately, few staff have sufficient time to 
spend with participants due to large caseloads, and keeping up with case file notes is 
even more difficult. 

Financial and Supportive Services 

Many participants in Section 401 programs face significant barriers in completing 
training, including the lack of adequate shelter, food, medical care, training, and 
childcare. In response to these barriers, grantees provide stipends or a variety of 
supportive services, or they refer participants to community agencies for these services. 
Site visits to 23 grantees showed that 22 out of 23 grantees in our sample provide 
supportive services with JTPA funds. (The one exception relied on tribal or community 
resources exclusively to pay for stipends and supportive services) . Below we discuss 
how these services are delivered. 

Financial Stipends 

Thirteen out of 23 case study grantees (56%) provide financial stipends to 
participants during training. These financial payments are made in the form of 
incentives, bonuses, wages, or as a needs-based payment. Where hourly payments are 
made, financial stipends ranged from $1.25 to $4.75 per hour of training; in other cases 
participants are given a maximum stipend amount during training— ranging from $30 a 
week to $1,500 per academic year for those enrolled in classroom training. The lower 
stipends of $1.25 to $2 per hour in classroom training are generally reserved for 
participants receiving other benefits or supportive services through the JTPA program 
or through other non-JTPA resources. 

Grantees varied in their policies for providing financial stipends. Typically, 
grantees provide financial stipends for participants enrolled in classroom training 
activities, including occupational skills training, GED or ABE programs, or classes in 
community colleges leading towards an Associate or Bachelor of Arts degree. Several 
grantees paid financial incentives to participants who complete job search preparation 
classes, most commonly, an introduction to the world-of-work program. Some 
grantees also provide financial bonuses ($100) to participants for passing GED exams. 
Programs also used stipends for one-time expenses such as books, school supplies, and 
tuition. 
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Among all case study grantees, the amount of payment was contingent upon 
attendance, and participants were monitored regularly for their participation in 
classroom training activities. Moreover, some grantees withheld financial payments if 
participants did not achieve a satisfactory GPA in classroom training activities. 

Arrangements to support the provision of financial payments varied among case 
study grantees. Sixteen case study sites actively leveraged non-Section 401 funding 
through coordination with tribal agencies, community-based organizations, or other 
federal resources such as the Pell Grant to supplement financial payments provided by 
grantees. Some grantees also provide some combination of stipends for training and 
other supportive services to meet a variety of basic needs during training. 

Supportive Services 

Supportive services can be very important in enabling participants to successfully 
complete the INA program, and, as was discussed in Chapter III, a variety of such 
services are routinely provided. Nationally, assistance for transportation, tools, 
equipment for training, clothing, and personal or family counseling are the most 
common forms of supportive services, and financial stipends or help in obtaining 
financial assistance is also very common. 

We learned that case study grantees offer similar supportive services as the 
grantees responding to the national survey. In the majority of case study grantees, 
some supportive services are provided either through the grantee, the tribe, or through 
referrals to other agencies. Transportation assistance is particularly common, and is 
provided in the form of bus passes, gas coupons, or mileage reimbursements. 

We also learned from case study grantees that some provide rental assistance in 
“extreme” cases. In these cases, grantees pay for some portion of rental costs for a 
short time until participants receive income provided through JTPA’s subsidized 
employment activities. 

Overall, the policies on how supportive services are used differed widely. The 
majority of case study grantees maintained very flexible policies towards who can 
receive supportive services and what they are eligible for. Several grantees for 
example, reported that supportive services are “up for grabs” for all participants 
enrolled in JTPA because grantees felt that any additional support was essential to the 
clients’ success. Most grantees, however, provide supportive services on a case by 
case basis or on an “as needed” basis based on an assessment of participants’ needs and 
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a review of program resources and resources within the community. Some grantees try 
to limit supportive services to participants who are already receiving other forms of aid 
from tribal or other non-JTPA resources. Several grantees even reported that they do 
not broadcast the availability of supportive services because they wanted to “save 
money” for direct training activities. 

To help fund what was reported to be a “very expensive” service, most case 
study grantees made an effort to supplement supportive services by identifying and 
leveraging non-JTPA funds to supplement participants’ financial needs. For example, 
over half of the case study grantees rely on tribal resources or other community 
agencies to increase supportive services, in the form of tuition assistance, added 
financial stipends, and child care assistance. In one local site, for example, participants 
who are enrolled in classroom training can receive financial stipends from the Bureau 
of Indian Affairs and child care assistance through the Child Care Block Grants 
arranged through the tribe. Additional supportive services from tribal resources are 
reserved for tribal members. 

The use of supportive services and financial stipends reportedly made a 
significant difference in helping participants remain in JTPA training activities and 
transition to self-sufficiency. Given the limited resources available, some grantees 
carefully targeted the types of supportive services and stipends offered to ensure that 
those most in need could receive adequate supportive services. A large number of case 
study grantees, however, have not developed specific strategies on how to make 
efficient use of limited supportive service dollars and have faced difficulties doing so 
because of their commitment to serving the needs of their clients. 

Summary 

This chapter highlighted the variation in the design and implementation of 
activities and services in support of training among our 23 case study grantees. We 
noted that some of the differences in how these activities were carried out were partially 
related to the differences in local contexts, such as whether programs were located in 
large service areas or were tribal or non-tribal grantees. 

In developing targeting strategies, most grantees faced difficulties in establishing 
specific targeting goals because they felt that all eligible IN A applicants were in need of 
services and the grantee thus did not want to give priority to some individuals over 
others. In local areas where targeting was an explicit goal, grantees targeted segments 
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of the population deemed “hardest to serve’” by developing clear procedures, such as 
implementing a “no-repeaters” policy. All case study grantees reported little difficulty 
in meeting their enrollment goals, and most relied on word-of-mouth referrals. 

Although this method is generally adequate, it can limit access to services to those who 
are not as well connected in the community’s networks. 

Most grantees were careful about tailoring assessment activities to individual 
circumstances. Accordingly, assessment practices varied widely, because of the 
differences in the backgrounds and needs of participants and program designs. 

However, in the majority of case study sites, assessment results often had only a 
modest impact on the kinds of services that participants received, because a variety of 
other factors and constraints came into play, including budget limitations, transportation 
barriers, and the unavailability of a full range of service providers. Moreover, clients 
often had strong opinions about what services they wanted and needed in the short-run 
(e.g., immediate income and subsidized or unsubsidized employment), often to the 
detriment of their longer-term needs. Case management services in general were 
practiced informally, and were often targeted to participants facing difficulty in 
training, with less attention paid to those who appeared “to be doing well. ’’ The trend 
towards “targeted” case management thus meant that some participants in need of case 
management did not receive it because they were unable to communicate this need. 
Effective case management is greatly hindered by large caseloads and large service 
areas. Some grantees have devised innovative methods to overcome these challenges 
by frequently communicating with instructors, worksite supervisors, and other service 
provider staff to identify clients’ needs early and intervene appropriately. And, 
invariably, case managers were dedicated and caring, and many demonstrated 
extraordinary efforts to ensure that their clients’ needs were met. 
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VI. BASIC SKILLS & OCCUPATIONAL SKILLS 
CLASSROOM TRAINING 



This section addresses formalized training which is intended to enhance basic 
academic and work- related skills, provide educational training leading towards a GED 
or high school diploma, or secure a post secondary degree or certification in a specified 
vocational or technical area. These services require attending classes and are focused 
on cognitive development and demonstration of sequential attainment of specific skills 
and knowledge. They are generally of longer duration than most other JTPA 401 
services and require self-directed commitment and motivation for participants to 
complete a prescribed course of study. The two types of training covered in this 
section are Basic Skills Classroom Training and Occupational Skills Classroom 
Training. 

Basic Skills 

Basic Skills Classroom Training is a service offered by over half of our sample, 
or thirteen of the twenty-three programs. Even so, it is the service option offered by 
fewer sampled grantees than any other IN A 401 service. Based on participant 
distributions for PY 96, of the thirteen sites, no grantees made BSCRT a first priority 
for service provision, only five grantees made it a second or third priority, while eight 
placed it as a fourth, fifth or sixth priority. This relative infrequency was a factor in 
our being able to observe only three BSCRT services over the course of our site visits. 

BSCRT, being a service which improves basic educational skills primarily in 
language arts and mathematics or prepares high school dropouts to pass a GED test, is 
by nature less sophisticated in content than the courses offered in the Occupational 
Skills Classrooms at community colleges and vo-tech schools. Overall, it was of mixed 
quality in teaching methodology, in facilities, and in teacher preparation. Of the 
thirteen grantees offering BSCRT in the examined sample, we observed three during 
on-site visits. These three were sites where the grantee was the provider, rather than 
enrolling students at accredited institutions such as state or tribally-operated colleges or 
vocational-technical schools. This often occurred not by design, but by circumstance. 

Adult Basic Education and GED training are often seen as the purview of other 
tribal departments, either as a program under the Education Department or as a service 
provided through other resources such as Headstart or JOBS. If an INA 401 program 
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does not offer BSCRT or has a limited service, eligible participants who are assessed as 
being in need of this type of remediation are often referred at intake to those programs. 
In some instances, local community colleges or vocational schools, either tribally or 
state-operated, offer ABE/GED training, and INA 401 participants are referred or 
encouraged to attend classes at those sites. In those instances, enrollment fees, books 
and tuition may be paid by tribal assistance programs or by the JTPA 401 programs. 
These eligible participants are offered other services as appropriate, usually in WEX or 
CSE to the extent their skill levels are adequate for placements. Some are therefore 
receiving ABE/GED training elsewhere while also being served by JTPA. 

When other resources are not available or adequate however, some INA/JTPA 
401 programs do offer basic skills ABE & GED services. In the sampled grantees of 
this study there was variance in the quality and extent to which participants could 
receive basic skills training. At one extreme, services were extensive and 
comprehensive, while, at another, they were minimal. In examining these services, 
four basic elements were reviewed and addressed. They are: 

• Types of providers, including who provided services, where services 
were provided, formal agreements, and how providers were selected; 

• Clients receiving services, including characteristics of those who were 
participating in ABE/GED programs; 

• Intensity and duration of services, including the length of training and 
the extent to which the time used afforded adequate learning 
experiences; and 

• Quality of training, including learning objectives, curriculum classroom 
interactions, instructor qualifications and methods of instruction. 

Types of Providers 

Each grantee has developed a unique way to provide ABE/GED training, and 
each is unique in determining how, when and where the services are accessed and what 
participants will be expected to learn. Service providers include state and tribally- 
operated community colleges, vocational-technical schools and in a few instances, high 
schools or alternative schools where remedial or GED classes are conducted. In the 
three sites observed, the INA 401 grantee itself was the service provider. Where that 
is the case, the classes are held at the grantee's facility, tribal office site or learning 
center in a specified classroom. Some participants at one site were also referred to 
learning settings in public school facilities. Those referrals occur because of over- 
crowding in the classroom or for the purpose of gaining additional skills training such 
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as the "world of work" basic job skills course, which is sub-contracted to an external 
private-sector provider at one site. Several factors seem to explain why grantees elect 
to be the service provider 1) it can be less expensive, 2) no other providers are 
available and 3) some participants seem more comfortable in an "Indian setting" or 
culturally-responsive setting than in the alternatives provided, particularly in urban 
settings. 

Basic Skills instructors at the sampled sites were either employees of the INA 401 
grantee or were instructors from nearby community colleges. Specific qualifications or 
special required training in adult education for instructors did not always appear to be a 
priority requirement. However, in all three observed sites, the instructors had achieved 
some post-secondary education and training. 

Clients Receiving Services 

The great majority of INA/JTPA 401 clients who are placed in the ABE/GED 
basic skills programs are high school dropouts. Approximately one-fourth of all the 
clients served in the three INA 401 sites were reported to have dropped out of school 
before graduation. The range is 21% to 28% of the total client base. Participants are 
encouraged to enroll in basic skills if they have neither a diploma nor a GED. In many 
instances, basic skills reinforcement is a required prerequisite for other services such as 
Occupational Skills Classroom Training or work-site placements. 

The percentage of BSCRT participants in the sample overall is relatively low. 

PY 96 participation data indicate that only one grantee, a rural tribal program, had 
more than 20% of participants placed in BSCRT ; this highest percentage was reported 
at 42%. Eight of the thirteen BSCRT providers in our sample, or the greater majority, 
reported that fewer than 10% of participants were served in BSCRT. One might ask, 
"why the disparity between reported high dropout rates of 25% in some service areas 
and the relatively low enrollment percentages?" There is no easy answer. What is not 
known is the extent to which dropouts come to the INA 401 JTPA programs for basic 
skills training or the extent that those in need of ABE/GED training are referred to 
programs or services other than JTPA. What we can say is that JTPA personnel state 
that clients are referred when the service is not offered by the INA 401 program. 

The actual numbers being served in the ABE/GED classes vary. It appears that 
three factors come into play which govern the numbers enrolled: 1) the funding level of 
the grantee (larger numbers are enrolled when the funding is larger) , and 2) the extent 
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to which there are alternate sources for the training, i.e., tribally-supported adult 
education programs or federally-funded basic skills services included in programs such 
as JOBS or Headstart and 3) the percentage of dropouts in the service population. In at 
least one instance, classroom participants were mixed, with JOBS, Tribal and 
INA/JTPA participants in the same classroom and using the same computer-assisted 
learning program. 

Most Basic Skills Classroom Training participants are older teenagers or young 
adults. One grantee program provides a high school completion course at a local public 
alternative school where participants may complete courses and graduate, or obtain a 
certificate through a proficiency exam. At another site, the high school graduates are 
described as lacking in basic academic skills due to the public school's deficiencies, and 
are enrolled in the IN A 401 program to enhance their reading and math skills. At two 
sites, an open-entry/open-exit policy causes the class sizes to fluctuate as participants 
come and go at their own pace. Participants are evenly divided between males and 
females at two of the observed sites and more predominantly female at another site. 

Intensity and Duration 

The three sites promote flexibility and accessibility through open-entry/open-exit 
policies. That is, students may enroll at any time during the year and may exit on 
completion of assignments/course, attainment of a GED, or high school graduation. 

All Basic Skills classes are described as self-paced with students progressing 
through assignments at their individual rates. In one instance, students are provided 
with workbooks consisting of drill and practice either in ABE basic skills or, more 
frequently, the standardized material to be mastered in preparation for taking the GED 
exam. In another instance, students work at individual computers progressing through 
a packaged Skills Bank III computer-assisted learning program. Under special 
circumstances, some also receive tutoring at a community college. 

At another site, classes are also self-paced but are structured in informal study 
groups with several volunteer teachers. This is enhanced with life-skills seminars and 
problem-solving activities to provide experiences beyond basic skills learning. 

Under the above circumstances, adult students must themselves be motivated to 
get the most intense training possible. While instructors serve as facilitators/mentors, 
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they do not formally structure the intensity, difficulty, or duration of the learning 
sessions. 

The structuring of time or duration of the training also varies from grantee to 
grantee. At one site, the training lasts for 4 hours a day, 5 days a week for a total of 
20 hours a week. Because of the flexible entry-exit policy, some participants attend 
class for several months while others attend a year or more depending on the individual 
student's progress. On the one hand, this policy provides students with the flexibility 
they need, given their other obligations; at the same time, without the expectation that a 
student will successfully complete the training in a given time frame, the student may 
not maintain the effort to completion. 

At another site, the learning center is open 4 hours a day, 4 days a week (with 2 
extra hours on Wednesdays for extended learning time). At the third site, there are no 
set hours, nor set schedule; participants "drop-in when and as long as they want to". 
Where there is some structure and high expectations are held by the instructors, the 
success rate is said to be at 80%, or as described by program staff, "most get their 
GEDs". Conversely, sites with a lack of structure led to limited success. 

At one site, the instructor seemed to only be "available as needed" and did not 
appear to monitor progress. This site was a minimally-funded grantee which paid part 
of the instructor's salary through the Section 401 grant and a student stipend of $1.25 
per hour. This site also had the highest percentage of high school dropouts. More 
intensive supervision would definitely have been desirable, but budget constraints 
severely constrained what the grantee was able to do. 

Quality of Training 

To evaluate the quality of training, we first examined the training content to 
ascertain if learning objectives were clear and clearly specified to participants. We 
then reviewed the curriculum and sought to discover how the curriculum was 
developed. In addition, an analysis of the context in which the content was delivered 
led to documenting how skills were taught and whether or not they were functionally 
connected to a "real world" environment. Documentation processes were also 
reviewed. 

Instructional methodology is the second major factor in the "quality of 
instruction" paradigm. Here we examined instructor/supervisor training and 
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background as well as observing opportunities for active learning, instructor and 
participant interactions, and the effective use of class time. 

Training Content. In all 3 sites observed, ABE training content was uniformly 
designed to bolster basic academic skills primarily in reading and mathematics. At two 
sites learning objectives were predetermined by computer software packaging; i.e. the 
Skills Bank and Skills Bank III adult basic education self-paced/self-testing programs. 
At the third site instructors determined learning objectives based on individual 
prescriptions derived from diagnostic pre-testing to determine deficiencies. GED 
training at all sites followed standardized learning objectives designed to prepare clients 
to take state-administered GED tests after being pre-tested at the INA/JTPA site. In 
addition, two sites expanded learning in the areas of creative writing, guest speakers, 
videos, field trips, JOBNET computer search and word processing. At one site the 
motivation to obtain a GED was the prospect of employment at a Casino. 

There appeared to be little connection to "culture" per se nor any need/attempt to 
make objectives cultural in nature. In general, expectations were clear as to why 
participants were there and what they were to learn. 

Curriculum Development. With the exception of the enrichment activities 
previously mentioned, curriculum scope and sequence was pre-designed either in the 
form of commercial computer-assisted skills programs or state/nationally designed 
workbook assignments designed to master skills required to pass a GED test. There 
seemed to be little deviation from the standardized content which GED candidates are 
expected to learn. 

Context in Which Content was Delivered. The ABE and GED skills were the 
content and the context. Although a variety of methods were used to transfer learning 
from the page or screen, the assignments themselves dealt primarily with improving 
those skills which are usually mastered at the high school level. The relationship to the 
"real world" or the "world of work" was not uniformly imbedded in the courses but 
more inferred - that is, "you can't get hired without a diploma/certificate". The 
absence of more explicit real-world connections appears to be a weakness in the sites 
we observed. 

Instructional Methodology. All sites adhered to the principal of "Instructor as 
facilitator" in theory. Instructors were there primarily to conduct assessments, answer 
questions, and administer and score tests, as students worked individually. They did 
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not, however, appear to actively facilitate the learning process. While there may have 
been other class discussions, only one "study group" was observed during our site 
visits. 



Instructional guidance was at a minimum at one site where the instructor was 
present only one day out of each week. This instructor drives from a local college to 
the reservation. It is planned that a second instructor from the local school district will 
also come to the class for two additional days. At the other extreme, at another site 
instructors were always present during regular hours. The use of volunteer instructors 
was also used at one site to increase student-teacher interactions. 

One is struck by the absence of participatory learning, as most of these observed 
programs had little or no classroom interaction in the learning process, either 
instructor-to-student or student-to-student interactions. One might surmise that the 
classes taught at the high school would be interactive due to the younger ages of the 
participants. Otherwise, the adult learners were expected to be independent learners. 
This element of instruction at some of the observed sites can be described as weaker 
than it could or should be. 

Occupational Skills Classroom Training 

We observed in-depth examples of Occupational Skills Classroom Training 
(OSCRT) in fifteen out of the twenty-three grantee sites in our sample. At one site, 
two Occupational Skills Training services were observed, and at another site three 
examples of OSCRT were observed, for a total of eighteen observations. With twenty- 
one out of twenty-three grantees in our sample offering this service, it is found in our 
sample on the same scale as Work Experience and Training Assistance. 

The capability to offer OSCRT is enabled and enhanced by the proximity and 
proliferation of state/tribal community colleges, private colleges/schools and state and 
tribal vocational/technical schools. With some exceptions, one or more of these 
institutions are situated within a reasonable driving distance to many participants 
regardless of the grantee's urban, rural, tribal or corporate status and regardless of their 
state locations. It is something of a testament to the states' and tribes' community 
college and vocational technical systems; they serve as invaluable resources for the INA 
401 grantees and for those individuals who seek post-secondary training/education but 
are not prepared to attend a four-year university degree program. 
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In addition, the course offerings at these institutions are broad in scope, offering 
a myriad of choices in the pursuit of training leading to a profession or vocation. On 
the community college level, there is also more flexibility, with courses of study which 
range in length from 1 to 2 years, resulting in certifications or Associate of Arts 
Degrees. An added incentive is the transferability of associate degree credit hours to 
four-year institutions, should a client choose to pursue a bachelor's degree. 

Unfortunately one barrier is often the cost. With INA 401 programs usually 
picking up the costs of tuition, fees, books and a living stipend over a period of 1 or 2 
years for each client receiving this service, the number of participants enrolled must be 
limited. There was a fairly uniform belief among INA 401 grantees that more clients 
could positively benefit from these services if there were more funds to accommodate 
them. In essence, the demand is greater than the available funds when INA 401 
grantees must balance OSCRT with other training and job services for which there is 
also a high demand. One factor which enables INA 401 programs to offer this service 
is that some tribes have other resources to assist with tuition and transportation, and, in 
some tribally-operated community colleges and vo-tech schools, tribal members are 
granted tuition waivers. 

Our evaluation of the OSCRT services addresses the same four elements reviewed 
in the Basic Skills Classroom Training services described earlier in this section. They 
are: 

• Types of Providers 

• Clients Receiving Services 

• Intensity and Duration of Services 

• Quality of Training 

In addition, we also reviewed and analyzed the Training Options which were 
available to participants at each OSCRT provider site. 

Types of Providers 

OSCRT services are provided on state-operated community college campuses, at 
state college/university extension sites, at local public vocational-technical schools, at 
local privately-operated business colleges and at tribally (or BIA) - operated community 
colleges or training programs. The extent to which an INA 401 grantee contracts with 
these entities to provide the OSCRT service varies based on the geographic scope of the 
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service area, on the educational/training needs of the clients, and on the course 
offerings at various provider sites. 

An INA 401 OSCRT service may provide multiple options, enrolling clients at 
any one of multiple providers. One INA 401 grantee simultaneously offered 
enrollment at a tribally-operated community college, a tribally-operated institute of 
technology, state community colleges in a metropolitan area, and two university 
settings in two different states. This was done to accommodate clients who lived in 
different geographic locations throughout an extremely large service area. Another 
INA 401 grantee contracted with universities, adult education centers, public and 
private schools and private post-secondary institutions/business schools. One grantee 
utilizes five state-operated community colleges located in its service area. Fewer 
options were provided by a somewhat isolated INA 401 grantee where a state college 
and a proprietary school were the two provider sites. The fewest options were offered 
by a tribal grantee which utilized the local tribally-controlled community college, 
described as "the only show in town" 

INA 401 grantees usually have formal agreements with the OSCRT providers 
which stipulate the costs for which the grantee is responsible. Separate contracts may 
be drawn up for each client. One INA 401 grantee indicated they had over 300 
classroom training contracts. 

The selection of providers often was based upon geographic proximity to clients 
and course offerings, which could meet client needs. Another strong factor, which 
surfaced frequently, was the longevity of the relationship between the INA-401 grantee 
and the provider; it was not unusual to hear that they had been "sending clients there 
for years". In those instances, the quality of the training was also a factor, with the 
grantees’ citing high levels of success with clients completing courses of study, and the 
high quality of staff provided at the institution. 

Reservation-based tribal INA 401 grantees consistently utilized tribally- 
operated/controlled colleges, technical institutes and training programs as well as other 
private and public institutions where available. At one tribal grantee site, the tribe 
provided training and apprenticeships at two tribal vocation-specific training programs 
(clerical and carpentry) as well as utilizing a state-operated vo-tech school. 
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Clients Receiving Services 

The overwhelming majority of OSCRT clients have a high school diploma or 
GED certificate, although a few clients lack either. OSCRT participants are assessed 
through various tests prior to enrollment as well. They usually possess adequate basic 
academic skills, with one site requiring at least a 10th grade achievement level in 
reading and mathematics and another stating that "most have a 12th grade reading 
level". Similarly, at one grantee site, clients must make at least 70% on a math test 
prior to enrollment in a Blackjack Dealing class. In addition, community colleges 
conduct pre-enrollment screening utilizing aptitude and interest testing instruments such 
as the TABE basic skills test, the COPS career survey, the VAL PAR basic skills test, 
and the ACET self-placement essay test. Clients must demonstrate acceptable levels of 
proficiency before they are accepted or enrolled. In most instances, if there are 
academic deficiencies, participants must complete remediation classes before entry into 
one-year certification or two-year degree programs. These tests are also used as 
guidance for enrolling participants in one training field rather than another. 

Thus, academic requirements for placement into OSCRT are more stringent than 
for placement in other INA 401 services. Moreover, OSCRT participants are described 
as highly motivated, in essence the "cream" of the INA 401 client-base academically 
and are "screened" into the programs. They are therefore considered to be likely 
candidates for success on the post-secondary level. 

The age ranges of OSCRT clients in our sample are reported to be between 18-55 
years at one site, 22-33 years at another site, and 19-30 years at another, with some 
average ages reported as 25 years and 37 years. Interestingly, several grantees 
indicated that, among their OSCRT clients, were those who were changing careers. 
Most were unemployed or underemployed. Many were single parents and most were 
economically disadvantaged. 

The numbers of clients served in OSCRT are predictably larger in those INA 401 
programs with larger budgets. At the largest program, one-fourth of the clients were 
served by way of classroom training, utilizing 15% of the program's total budget. At 
another, 14% of the total program budget covered clients' OSCRT expenses, up to 
$5,000 per student. The overall range was from 3% to 31% of total INA 401 program 
dollars. In terms of numbers of clients served annually in OSCRT, the range at sample 
sites was from 609 at the largest site to approximately 8 at a small program site. 
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The gender of clients did not play a role in OSCRT placements in the overall 
sample. At some sites most clients were female, at some the majority were male, and, 
at some, they were equally divided. Interestingly, eleven women were enrolled at an 
Aviation Training Academy in a program where the client base was described as 
"mostly men in their mid-twenties". In building trades occupational training, the 
clients were predominately male. Overall, there is a tendency for more females to be 
enrolled in OSCRT than males. This may be related to economic factors (i.e., 
alternative support for child care) and to the availability of jobs in clerical fields, 
nursing fields, and day care workers — positions more frequently filled by females. 

The distance to be traveled to attend classes was a factor mentioned by several 
INA 401 program personnel. As might be expected, clients who had to move to a 
different city or commute significant distances were at more of a disadvantage than 
those who could be placed at institutions/schools closer to their residences. This was 
more of a problem in the more remote reservation-based tribally-operated programs 
than in the urban settings. 

Fortunately, some colleges and universities had established extension classes on 
the reservations, a promising practice but limited in scope to a few courses of study. 
Negotiating these kinds of arrangements on a broader basis could alleviate the problem 
but not eliminate it entirely. 

This is probably a good place to address the sense of community, which was 
voiced by many clients. Their responses centered on two issues, one being their desire 
to stay in their own communities due to family responsibilities and support systems, 
etc., and one being their desire to learn an occupation which would provide 
employment in a local market. The later reflects the sense of "giving back" or "helping 
my people" as a way of maintaining community /tribal connections. 

As to cultural adaptability, all OSCRT clients were proficient English speakers, 
some being bi-lingual native language and English speakers. The degree to which they 
were "traditional" or "assimilated to the larger culture" was a variable, as we 
encountered individuals across an acculturation scale from one end to the other. 

One is struck by several positive characteristics of OSCRT clients. While they 
possess high school diplomas or GED certification, most are described as under-skilled 
for the job market to which they aspire. However, they are described in terms such as 
"prepared", "determined", and "highly motivated". In general, they also possess 
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perseverance; one participant stated that she waited two years for an OSCRT opening at 
her INA 401 program so that she could pursue a nursing career. 

Training Options 

Across sampled programs the training options available to INA 401 OSCRT 
participants is broad indeed. In describing them, we will attempt to delineate the 
courses of study, the degree/certification options, services provided in support of 
training, and any additional training options which might accompany the Occupational 
Skills Classroom Training provided at various provider or grantee sites. 

Accessible resources (or accessible service providers) is the main factor in 
determining the options to be offered by an INA 401 program. In addition, funding of 
the OSCRT service at each program is also a determinant. The more resources and 
funds available, the more options are possible. 

Looking at the categories of providers, one can summarize the options that are 
available. Providers may be categorized as follows: state-operated vocational-technical 
schools; state-operated community colleges; tribally-operated community colleges, 
technical institutes and training programs; private post-secondary technical/business 
schools; and college or university satellite/extension campuses. 

1) State-Operated Vocational-Technical Schools offer courses of study which, 
upon completion, lead to certifications established by state boards. Depending on the 
vo-tech school in question, participants may obtain training in any of the following 
areas: Health Service Technology, Air Conditioning, Allied Health, Auto Collision 
Repair, Auto Mechanics, Business and Information Processing, Carpentry, Child Care, 
Drafting, Electronics, Food Service, Horticulture, Physical Therapy, Precision 
Machining, Printing, Graphic Arts, Truck Driving and Welding. We were able to 
observe a Health Science class at a state-operated vocational-technical school. The 
students were all female. 

2) State-Operated Community Colleges offer one- or two-year courses which, 
upon completion, lead to state certification or Associate in Arts degrees, with 
transferable credit hours accepted towards a bachelors degree at four-year universities. 
In some cases, there also seem to be shorter courses of study for aides in various 
occupations. 
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If the range of options is wide at vo-tech schools, it is often wider and in more 
depth at most state-operated community colleges. Offered are vocational/technical job 
skills development courses for certification, which mirror the offerings at vo-tech 
schools. In addition, one- and two-year courses of study are offered in areas such as: 
Law Enforcement, Cosmetology, Nursing, Respiratory Therapy, Applied Science, 

Food Service, Child Care, Computer Technology, Accounting, Natural Resources, 
Environmental Science, Optical, and Culinary Arts. We were able to observe a nursing 
class at a state-operated community college. The class consisted of Indian and non- 
Indian students. At this site it was noted that INA-401 enrollments/training services 
were based upon the state's economic job market analysis, a practice which increased 
the participants' ability to obtain employment after completing the training or course of 
study. 

3) Tribally-Operated Community Colleges, Technical Institutes and Specialized 
Training Programs offer a broad range of courses. Similar to their state-operated 
counterparts, the tribal community colleges observed offer both certification and 
associate degree programs in both vocational and technical areas of study, while the 
tribal technical institutes focus predominantly on the more technically-oriented 
disciplines (i.e. electronics, drafting, surveying, engineering). At one site the tribe 
provided certification programs through its own auspices in carpentry and office skills. 
We were able to observe a Business Computer Applications class and a Blackjack 
Dealer class, a Carpentry class, and an Office Skills class. In the Blackjack Dealer 
class the students were Indian and non-Indian, some were INA participants and some 
were not. We also observed a Computer Literacy class of six females and seven males, 
a Nursing class which was said to be usually 80% female, a Graphics Arts class where 
6 INA-401 students were described as "highly motivated," and a Business Skills class 
where participants' eligibility was determined by the Social Services Department based 
on their disadvantaged status. 

4) Private Technical/Business Schools offer certification programs in areas such 
as: Business Skills, Computer Training, Application Development, Microsystems, 
Administrative Support, Keyboarding, and Data Processing. Some of these private 
schools provide highly specialized training, such as the National Security Field 
Experience Initiative Curriculum at the Livermore Lab in California and the Associate 
of Arts Degree in Applied Science in Aviation Maintenance (and certificates in 
Airframe and Power Plant Technologies) at the Aviation Training Academy in 
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Indianapolis. We were able to observe classes at the Aviation Academy and a 
Keyboarding class taught by a private computer processing company. 

5) College/University Satellite/Extension Campuses were training sites operated 
by state institutions of higher education located on or near reservations or in urban 
areas. These extension classes and courses of study were similar in content, although 
the training options were not as varied as those found on the home-based campuses of 
community colleges. At the site we observed, the tribe provided the space for the 
study of Office Technology. This was the most unusual situation, because the tribe 
virtually bought the course of study from the college for $70,000 and brought it to the 
local community. Such a creative solution is to be encouraged if the expense is not 
prohibitive. 

Services Provided In Support of Training and Other Options 

The participants' costs for the OSCRT Training are provided by INA-401 budgets 
primarily. These costs include tuition ranging from $500 to over $5,000, as well as 
costs for books, supplies, fees and frequently, living stipends. In addition, at many of 
the provider sites, supportive services are provided, such as counseling, career 
guidance and financial assistance resources. A few provide child care. Three IN A 401 
programs indicated they offered assistance with transportation costs as well. Overall, 
every effort seems to be made to ensure that clients are given the support they need to 
succeed. 

Described at one site as a "full range of support", supplementary classes are also 
provided in the form of brown-bag (lunch) discussions, basic skills remediation where 
needed, extensive placement assessments, "world of work" training, job placement 
services, tutoring, field trips, and, in one reported instance, incentive awards. 

In summary, clients seeking training in Occupational Skills, be they vocational or 
technical in nature, are more often than not offered sufficient, even abundant, options 
from which to choose. Not all INA-401 programs are equal in this regard. However, 
given the constraints of geography and available funds, they are responding in creative 
ways in order to meet the needs of their clients. 

Intensity & Duration of Services 

In examining the intensity and duration of Occupational Skills Classroom 
Training, we analyzed the comparative length of time it took to complete the course of 
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study as well as the amount of time spent in the classroom. In a few instances, those 
activities offered outside of the "formal" classroom are also taken into consideration. 

As to duration, or the length of time required to complete the training, participant 
students could be found enrolled in short-term courses which lasted four to fifteen 
weeks, medium-term lasting six to nine months, and long-term courses which lasted 
one or two years to reach completion. Within this framework, there was variability as 
well. For example, those participants who were enrolled in Associate of Arts degrees 
were expected to complete courses over a two-year span, usually having to meet a 
prescribed number of contact hours per week throughout the 2-year curriculum. 
Vocational courses such as Nursing required 73 credit hours, spanning four semesters 
plus one summer session at one site. The Health Sciences Technology degree, a 2-year 
program, requires 1,026 hours of instruction over 18 courses. 

Technical courses of study resulting in certification might be one year in length or 
require two or three semesters for completion. Course duration in community colleges 
and vo-tech schools and business schools usually are on a trimester or quarterly system, 
which allows for flexibility in scheduling year-round and facilitates an open-entry/open- 
exit policy which accommodates the schedules of adult learners with family or work- 
related responsibilities. University extension and satellite campuses offer even longer 
courses of study; at one site the Nursing curriculum took 36 months to complete. 

A few INA 401 grantees limit the time that a participant may be served in 
OSCRT, but most indicate that the client will continue to receive the training support as 
long as he or she maintains a specific grade point average and acceptable attendance 
rate leading towards a certificate or degree. This seems to provide extra incentive for 
clients to successfully complete the courses. 

As to the amount of time spent in the classroom or lab activities, where provided, 
again there is a wide range of requirements. Some classes meet one day a week for a 
few hours or a full day, some meet up to five days a week from 8:00am to 
4:30pm/5:00pm. Within these two extremes are variations in numbers of days and 
hours of classroom and laboratory hours required. Students are required to attend 
classes with limitations on the number of absences allowed before penalties or a faili ng 
grade results. 
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Generally speaking, duration and intensity are carefully structured with some 
flexibility built in, which allows adult learners to maintain their class schedules and 
courses of study. Should a student have to drop out, they are encouraged to return and 
resume their study within a reasonable time. 

Quality of Training 

One way to judge the quality of OSCRT training is to look at the reported 
completion and placement rates. While we do not have data on all sites, the rates given 
to us are extremely positive. Five sites report placement or employment rates in a 
range of 73% to 94%. Additionally, three other sites report completion rates at 95%, 
attesting to their success. To reiterate, our assessment of the quality of training 
included the review of classroom materials and observations of classrooms, in order to 
ascertain information on the following elements of quality: 

• Training Content, including clarity of learning objectives and clarity of 
specification to participants. 

• Development of curriculum 

• Context in which the curriculum was delivered. 

• Instructional methodology, including opportunities for active learning, 
instructor-participant interactions and effective use of time. 

Training Content. Training content refers to the skills to be acquired and the 
knowledge expected to be learned in the classes observed. Some overall course content 
as it was made available to us is also included, where it sheds light to the scope and 
sequence of the curricula offered at the provider sites. 

Three categories emerge under which we can classify the classroom observations. 
First is in the area of business (technology, computer applications, etc.), where we 
observed seven classes. Second is the health occupations area (3 classes observed). 
Third is the specialized vocational area in which we include classes observed in 
Carpentry (2), Aviation Maintenance (1), Blackjack Dealer (1), Graphic Arts (2), and 
Career Preparation (1). 

Business curricula and objectives are practical, applicable and focused on specific 
skill development. While there are underlying theoretical foundations inherent in the 
content, assignments are designed for practical application in a working setting. This is 
true whether the INA 401 participant is enrolled in a private, state-operated or tribally- 
operated post secondary setting. Six of the seven classes observed in the business field 
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were primarily training in computer technology /applications/keyboarding. The 
seventh, a course in office technology, contained computer applications as well as 
accounting and career development. Computer literacy in various word processing 
applications, keyboarding, spreadsheets, desktop publishing, and other elements of 
computer mastery comprise the skills learned in these classes. 

In all cases, the learning objectives are imbedded in the textbooks and exercises 
in the classroom as well as in homework assignments. Frequently the computer classes 
also include professional development and career education training which purport to 
provide instruction in cooperativeness, diligence, leadership, communication skills, 
problem-solving, job interviewing, poise, dress, goal setting, job searching and resume 
writing. The intent is to provide a work-ready graduate who possesses office skills and 
personal skills. The training objectives are established at the provider site and are 
standardized to conform to the certification or degree programs' requirements. It 
seems that most of these courses are thus focused on facilitating the transition into the 
"world of work" . Participants follow a specified study plan either dictated by the 
text/course of study or individually prescribed. In either case, there is clarity in the 
specified objectives and presumably objectives are clearly understood by participants. 

In three of the classes, American Indian cultural elements are also a part of the 
content and practices in the classrooms. They are said to be "designed for the native 
learner in mind" and incorporate their cultural values, concepts, and native language 
where appropriate. In addition, multicultural or cross-cultural issues are explored in 
seminars at one site. At another, cultural field trips and guest lectures enrich the 
curriculum. 

In the three health-related classes we observed, content was medically based and 
specific to the day-to-day practical application in hospitals, health clinics and doctors 
offices. We observed two classes in Practical Nursing and one in Health Science 
Technology. Nursing students, as required by state certification, were required to 
master the content in multiple textbooks as well as in class and laboratory work. The 
Health Science Technology class followed the state-certification curriculum required for 
nursing aides, home health nursing or practical nursing. The learning objectives were 
specific, clear and sequential. Participants appeared to understand what they were to 
learn. 
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In the observed vocational classes of Carpentry, Aviation Maintenance, Graphic 
Arts and Blackjack Dealer, highly specialized curricula guided the sequential attainment 
of learning objectives. Mastery of the learning objectives provided a clear path to 
successful completion of the courses. They were characterized by formal syllabi and 
included textbook, audiovisual, and hands-on learning. When required, participants 
applied learning in labs, machine shops, at blackjack tables, and at drafting tables. In 
addition, internships were a part of the learning experience. Participants intern at such 
places as US West and Lawrence Livermore Laboratories. Again, participants were 
aware of their learning objectives and the curriculum requirements. 

Development of Curriculum. Curriculum includes the basic content covered in 
the INA 401 OSCRT courses as well as any enrichment activities beyond the content. 

In all the courses of study that lead to state certification, curriculum materials have 
been developed to ensure that graduates are able to pass the state exams. This involves 
the use of standardized texts and workbooks, which are sequential and incorporate 
skills-based and cognitive learning. For the most part, these texts are commercially 
created and the majority of providers use texts developed elsewhere as a basis for the 
course work. Class and homework assignments also tend to follow the scope and 
sequence of the curriculum design. 

Beyond the core knowledge gained from texts in the various vocational and 
technical areas, instructors frequently broaden the curricula to include field trips, guest 
speakers, audio-visuals, career searches and group discussions and lectures about work 
habits, environments, expectations and local job-market information. 

The OSCRT instructors, particularly in accredited institutions, usually have at 
least a bachelor’s degree with additional experience in the professional/vocational areas 
they teach. They are experienced to the degree that they are able to tailor the curricula 
to their locales and to the individual needs of their students. At two tribally -operated 
provider sites, instructors design curriculum contents based upon community and 
employer surveys. 

Curriculum contents are delineated in course syllabi, which outline the reading, 
and workbook assignment as well as individual and group projects required in the 
various courses. Most instructors describe their curricula in terms of being somewhat 
flexible or adaptable, and state that they revise them to stay current with technology. 
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regulations or other requirements. Most also indicate that their course standards are 
guided by employer needs, seeking to ensure that graduates are work-place ready. 

Often courses require hands-on skill development, such as computer technology, 
nursing, carpentry, aviation maintenance, etc., and include activities in "real world" 
settings, such as computer labs, construction sites, and airplane facilities. Three of the 
sites indicate that students are placed as interns at actual worksites such as the 
Livermore Laboratory where they study the National Security Field Experience 
Initiative curriculum leading to certification in the handling of nuclear/hazardous waste 
material. 

While the curricula are uniformly pre-set, INA 401 participants undergo 
comprehensive assessments prior to placement, are provided tutoring if needed (math at 
two sites and physics at one site) , and are guided to adult education classes for other 
basic skills reinforcement if required. 

In addition to textbook/workbook driver curricula, computer-assisted learning is 
predominant in office-skills/computer technology classes and is frequently designed 
according to self-paced, self-directed tutorials found in various software packages. 

Context In Which Curriculum was Delivered. A contextualized OSCRT 
curriculum is one in which relationships are drawn to the "real world" or to the 
applicability of the contents to the actual experiences of the vocational work 
environment. We also include the extent to which the contents are contextualized in 
terms of cultural appropriateness to the Native American client. The context is 
observed either in the content of materials covered, in the activities designed to 
accompany content, and/or in the personal interactions in the classrooms of OSCRT 
services. 

We will first address the extent to which "real world" applicability is imbedded 
within the observed curricula of our sampled OSCRT service. Several factors are 
indicative of contextualization regarding preparation for the workplace environment; 
they include personal skill building, professionalism, applied skill building, and 
internships. 

Personal skill building may be defined as including instruction, discussions, or 
activities that enhance the client's ability to function as an employee in a work setting. 
At least 47% of the OSCRT providers were observed or reported to include training in 
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such areas as decision-making, problem-solving, critical thinking, communication skills 
and working cooperatively. The last, cooperativeness, was especially prevalent overall. 
Almost all sites exhibited some type of teaming or group work/projects in which 
participants were required to work together to achieve an objective or class assignment. 
Since current prevalent thinking encourages group cohesiveness and cooperative work 
behaviors, these INA 401 clients are well served. All OSCRT classes should be 
encouraged to include this training. 

Professionalism may be defined as training that enables clients to be more aware 
of their roles as employees and understand the worksites' employee requirements. We 
here include training by way of class discussions, lectures and supportive services at the 
OSCRT provider sites. Elements of professionalism training observed, or reported to 
us, include punctuality, responsibility, dependability, courtesy and respect. We also 
include classes or sessions with counselors which address "world of work", resume 
writing and career/job search training. This was one of the strongest of the 
contextualized elements of curricula, with 82% of the INA 401 OSCRT providers 
including part or all of the above elements. It is possible that all of the providers 
addressed one or more elements. 

Applied skill building contextualizes the learning process by allowing the 
participants to experience hands-on application of the concepts learned. Whether it is 
using computers in a computer lab, working with the tools and materials in a 
construction class, or demonstrating mastery of nursing techniques in a classroom or 
laboratory setting, students are actively engaged in performing "real world" tasks. This 
was another area where INA 401 OSCRT providers were especially effective. Eighty- 
two percent (82%) of the classes observed, or reported, hands-on activities in which 
students performed tasks, under the supervision of the instructor, at least twice a week 
or daily in some instances. Again, there is the possibility that hands-on activities exist 
in all OSCRT classes but were not observed or reported. 

Internships are those instances where INA 401 OSCRT clients were placed at the 
worksite after having received some training or where they received all of the training 
at the worksite. In these instances, training resembled On-the-job Training experiences 
but was designed to follow a training curriculum rather than perform a job per se. 
Clients were classroom trained and supervised by employees at the worksite. This 
approach was found at only 4 OSCRT sites or 24% of those observed. It is an option 
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to be further explored in those instances where highly specialized training is required or 
adequate teaching capability coincides with worksite operations. 

The final element examined was the extent to which American Indian culture was 
infused in the curriculum as a form of contextualizing the learning experience. Where 
training was specific to a tribal enterprise, such as the Blackjack Dealer class, elements 
of modern culture in the form of tribal operations, mores, expectations etc., were 
inherent in the day to day training. We were able to identify 6 or (35%) of the OSCRT 
classes that infused cultural values, beliefs or native language in the curriculum. Two 
purported to have designed the classes "with native students in mind,” and one 
addressed the cultural attributes of harmony, beauty, balance and unity. These six 
OSCRT services were provided by tribal/native-controlled technical institutes and 
training centers. Among the remainder, some cultural references or "understood" 
elements were present but not specifically designed into the curriculum other than field 
trips or guest speakers. The extent to which culture plays or should play a 
contextualized role is the judgement of individual INA 401 programs. 

Instructional Methodology. Classroom observations were the primary ways we 
gathered information to evaluate the effectiveness of instructional methodology in the 
INA 401 OSCRT services. Here we were seeking to learn the degree to which clients 
were actively engaged in the learning process and how extensive those learning 
opportunities were. We also observed the frequency and nature of interactions in the 
classroom, both instructor-to-student and student-to-student interactions. Finally, we 
observed and questioned how effectively time was being used inside the classroom and 
outside the classroom, where applicable. 

Our site visit time frames allowed us to observe classrooms for one hour or less 
at each site. We therefore are providing "snapshots" , which appear to be indicative of 
the INA 401 OSCRT learning dynamic overall. 

Were clients actively engaged in the learning process? In 88% of observations 
(15 out of 17), clients (students) were engaged in active learning all or part of the time. 
That is, they were working individually or in groups/teams to complete assignments or 
projects. They were engaged at computers, were actively building or designing, were 
demonstrating learned concepts or creating their own portfolios. As for the two other 
sites, we were able only to interview the staff at one site, leaving only one site where 
active, participatory learning was lacking. Instructors at the 15 positive sites appeared 
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to endorse and value the outcomes when active learning accompanied lectures or text 
comprehension. This type of learning is also characterized as "hands-on" learning and 
usually involved the manipulation or use of objects and machinery, including tools, 
computers, nursing paraphernalia, cards, office equipment, first aid supplies, print shop 
equipment, and computer software. 

Opportunities to learn were extended beyond the classroom as well. Almost all 
courses included homework assignments at least once a week and some had homework 
assignments daily. Additionally, individual and small group tutoring was provided, as 
were extended hours of class time at several sites, to ensure adequate or additional 
learning time for students who needed it. One class employed the use of "study 
buddies" for reinforcement of learning. 

Related to active learning are classroom interactions. What role does the teacher 
or instructor play? How do students interact with the teacher and with one another? 
How extensive are the interactions? These were questions we sought to answer through 
our observations. In at least 76% of our observed OSCRT classrooms, interactivity 
was a dominant method of instruction. Teachers described themselves as 
facilitators/mentors and described their classes as participatory. Most stated that they 
lectured only a portion of the time or "little at all", preferring to let students interact 
rather than passively listening. Lectures were accompanied by demonstrations or 
modeling of concepts by the teacher, followed by students questioning and then 
demonstrating their mastery individually or in groups. Discussion groups were 
prevalent, as were team and group activities. 

In addition, teacher roles included daily monitoring, administering oral, written 
and skills tests and providing feedback to the students. The quality of interactions can 
be described in terms such as "lively" "easy-going" "supportive" and "cooperative". 
Learning environments for adult learners are said to be more successful where the 
students take ownership of their learning and are in an environment that is 
psychologically comfortable, lacking in intimidation. With the exception of one site, 
INA 401 OSCRT providers do appear to employ facilitative instructors who adhere to 
those criteria. Many were described as "caring" about their students. This was true 
regardless of the type of provider, the locale, or whether the instructor was Indian or 
non-Indian. 
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The structuring of class or learning time was the final element of our analysis. 
How was class time used and was it effective? By all indications previously described, 
we can conclude that learning time in OSCRT classes is effective overall, that students 
are not wasting their time or the instructor's time. In most courses, learning time is 
divided between classroom instructional time and laboratory or clinical setting 
activities. These are frequently scheduled on alternate days, similar to university time 
structures, or scheduled for one to occur in the morning (lecture) and one in the 
afternoon of the same day (lab) or vice versa. 

The terms self-directed or self-paced are frequently used to describe how time is 
structured as students progress through assignments and projects. This is another way 
that students are given the responsibility for their own learning and for working 
cooperatively on group projects. It is probably harder to let one's own work slide 
when there is inter-dependency for accomplishments and reward. 

Class time and "extra-curricular" time is enriched through the use of videos, 
films, field trips and guest speakers at many sites. At least 65% (11 of 17) of the INA 
401 OSCRT service providers build in time for clients to participate in these activities, 
which add depth and dimension to their learning experiences. 

In summary, the two types of educational classroom services, Basic Skills and 
Occupational Skills stand in contrast to one another. The first is the provision of 
fundamental training which is a prerequisite for functioning in the vast majority of 
occupations, namely reading, writing and calculating and obtaining the equivalent of a 
high school diploma. The other, the training needed for specific occupational 
employment, is professional in nature and usually accompanied by certification and 
college credit. Is one more important than the other? Maybe not. Is one more 
emphasized throughout the sample? Yes. Occupational Skills. Occupational Skills 
Training outcomes are more obviously positive— most participants completing courses 
of study in chosen fields— most of which coincide with needs in the labor market. 

They are therefore, more likely to obtain long term unsubsidized employment. 
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VII. QN-THE JOB TRAINING 



This chapter focuses on the On-the-job-Training (OJT) component of the JTPA 
Indian and Native American program. The objective of this training activity is to 
provide JTPA participants with work-based opportunities for occupational skills 
training after employers have hired them. This “hire-first” strategy is intended to 
induce employers to provide quality skills training for participants who are in-need of 
such training in order to maintain their employment status. Employers may be 
reimbursed for up to 50% of participants’ wages for up to six months based on the 
terms of individual contracts negotiated by employers, participants, and JTPA staff. 
Reimbursements are intended to compensate employers for productivity lost as a result 
of participants’ training— without which participants would be ineligible for 
employment. Although OJT is primarily a training activity designed to promote the 
acquisition of occupational skills, there is an expectation of long-term, full-time, 
unsubsidized employment after the contract expires. 

The chapter will discuss findings generated by site visits to 23 grantees and in- 
depth observations of OJT programs at 6 of these sites. First we provide an overview 
of the OJT service activity, including a discussion of statutory requirements and 
objectives. We then describe the objectives of the grantees that provide OJT programs 
as a part of the services packages they offer their participants. We also describe the 
design and delivery of OJT programs, including challenges and successful practices. 
Finally, we use the model of high-quality training (presented in Chapter I) to assess the 
effectiveness of the OJT component of the JTPA IN A program. 

Overview of OJT 

On-the-job-Training is an activity designed to link employers hiring in specific 
occupational areas and individuals who command basic skills and appropriate core 
knowledge about these occupations, but require training in specific skills or 
competencies. OJT is also an allowable service activity outside of the Title IV 
program. However, two factors distinguish INA OJT programs from other types of 
work-based job training programs. First, the dearth of employers located in or near 
most Title IV services areas, especially on reservations, prevents many grantees from 
developing OJTs with private sector employers. Thus, these grantees often use OJT to 
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provide training opportunities for newly hired tribal agency staff or for individuals 
seeking training, and subsequent employment, with tribally-owned enterprises. 

In addition, because the JTPA IN A program seeks to enhance the economic well- 
being of not only participants, but the communities from which they come, INA 
grantees serving areas that do offer many employment opportunities often target Indian- 
owned businesses as potential work or training sites for participants. These grantees 
perceive the OJT program as a means of helping Indian firms defray some of the costs 
associated with training and hiring new employees and providing needed support for 
growing Indian businesses. Presumably, as these businesses prosper, they will enhance 
the vitality of the Indian communities in which they are located, both by providing 
employment opportunities and offering goods and services to local residents. Grantees 
perceive this support for Indian-owned firms as well within the mandate of the JTPA 
Indian program. 

On-the-job Training activities under Title IV are designed to provide participants 
with meaningful training that will enable them to learn the skills required to perform 
effectively in their new jobs. For employers, the program provides opportunities to 
hire long-term employees who receive customized occupational training provided 
during work hours in exchange for a subsidy to compensate for the presumed loss of 
productivity during such training. Participating employers are expected to work with 
JTPA staff to develop contracts, in which key features of the training plan are 
identified, such as length, duration, and objectives of training. Participants, 
employers, and JTPA staff are also expected to describe when and how participants 
seek to accomplish their training goals. The training plans are intended to help 
employers identify the areas in which participants require new skills, plan to deliver 
appropriate training in these areas, and ensure that participants’ training programs will 
enable them to achieve measurable skills gains by working toward specific training 
objectives. 

Public and private-sector employers may participate in OJT programs. The 
contracts are binding for the duration of the training program— usually to a maximum 
of 6 months. Participants typically engage in program activities on a full-time basis, 
but the proportion of time spent training and the corresponding proportion of time spent 
working vary significantly with each OJT contract. Unlike other work-based activities 
under the Title IV program, OJT programs generally target participants who have a 
solid work history or who have already completed another training component. Typical 




VII-2 



176 



participants are work-ready high-school graduates, and may be skilled or semi-skilled 
in a particular vocational area. These participants are generally compensated at an 
hourly rate that exceeds minimum wage. For this reason, OJTs are perceived as 
“expensive” program activities by grantee staff. Finally, because participants are 
actually hired by their employers prior to developing the OJT contract, there is a clear 
expectation of continued employment after the contract has expired. However, 
ensuring that OJT programs do, in fact, lead to long-term, unsubsidized employment 
for participants has been a challenge for Indian grantees. 

Fourteen of the 23 sites we visited as part of the evaluation offered OJT as a 
service activity. At these sites, we conducted interviews with administrative staff at all 
levels to learn about program design, delivery of services, and outcomes. At six of 
these sites, we were able to observe service activities and conduct interviews with OJT 
participants, their employers and case managers, and the administrative staff of the 
INA programs responsible for OJT services. We also reviewed the case files of many 
of these participants. During our site visits to the ten grantees that did not offer OJT as 
a service activity, we held discussions with program administrators and front-line staff 
about why they had not developed OJT programs or why they had eliminated the 
activity from the menu of services they provide. 

In this chapter, we discuss the design and delivery of OJT service activities based 
on our site visits to the 23 grantees in our sample. First we discuss how and why 
grantees emphasized (or did not emphasize) OJT as a service option, and identify 
common features of OJT programs across sites. We then address the diverse objectives 
of OJT across grantees, examining the ways in which local program designs address the 
needs of Indian participants and communities. Finally, using the quality principles 
identified in the model of high quality training introduced in Chapter I, we provide an 
overall assessment of the OJT program component. 

OJT Frequency and Degree of Emphasis Across Sites 

Fourteen of our 23 case-study sites, or 62%, offered OJT as a service activity 
during the programs year in which we visited them. 1 Two additional sites reported 
that they had offered OJTs in past years, but had ceased using them in response to 



1 However, sites offering OJT services were somewhat over-represented in our sample. Based 
on PY 95 ASR data, only 40% of all grantees reported offering OJT services to their participants. 
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federal budget cuts over the past three years. One of the 14 grantees, a tribal grantee, 
indicated that OJT had not been an area of focus for several years, but as a result of 
efforts to conduct job development with target industries, OJT contracts were again 
becoming an attractive service option. 

Among the grantees in our sample, tribal entities were more likely to offer OJT 
services to their participants than non-tribal grantees. Of our 23 sample sites, 14 (or 
60%) represented tribal entities, and 9 (or 40%) represented Indian-serving 
organizations with no specific tribal affiliation. 2 Among the tribal grantees, 10 (or 
71%) offered OJT services to their participants; while among the non-tribal grantees, 
only 4 (or 44%) did so. 

This finding was somewhat surprising because non-tribal Indian-serving 
organizations tend to provide services in areas that are more urban, and therefore offer 
more opportunities for partnering with private-sector employers. We anticipated that 
OJT programs would represent a larger share of these grantees’ total services than they 
would among tribal grantees, which offer fewer private sector employment 
opportunities because of the remoteness of their reservations. This was not the case. 
The examination of the service designs of grantees serving diverse populations in 
different contexts, provided later in this chapter, will shed additional light on this issue. 

Reasons for Not Using OJT 

Among the nine sites that do not currently enroll participants in OJT programs, 
the following were cited as the key factors that prevented their use: 

• Inadequate funding. Almost all of these nine grantees indicated that 
their current funding levels prohibited the development of OJT 
programs. Two of them reported that they had offered OJT service 
activities in prior program years, but could no longer afford to do so. 3 



2 Although many of these grantees served participants from a single or only a few Tribes, 
depending upon the diversity in their local service areas. 

3 However, among these grantees, all but one offer either WEX or CSE, and more than half of 
them offer both, even though these service activities have the potential to cost as much or more per 
participants as OJT programs. 
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• Absence of private sector employers willing to hire OJT participants. 
Although the lack of private sector employers represents a challenge for 
most grantees, three grantees 4 specifically cited this issue as a barrier 
that prevented them from offering OJT service activities in particular. 5 

• Potential for employer abuse. Most sites reported that they had 
experienced or were aware of instances of employers taking advantage 
of OJT programs and participants. 6 Three sites reported that they did 
not offer OJT specifically to avoid such abuse. In one site, this policy 
was informal— OJT existed as a potential service activity, but program 
staff had not enrolled a participant in an OJT program for several years. 

• Low demand for OJT service activities among participants. Several 
grantees reported that they had made strategic decisions to omit OJT 
service activites from their program designs because other service 
activities were more appropriate for their particular customers. One 
grantee specifically pointed to the lack of interest in OJT services 
among participants in the local service area. This grantee reported that 
the vast majority of participants required both basic and occupational 
skills training— the grantee, therefore, emphasized these services rather 
than work-activities or work-based training. 

Overall Benefits and Challenges in Using OJT 

Data from the PY 96 ASRs show that OJT represented a fairly small percentage 
of the total cost of services provided to participants among most of the 23 grantees in 
our sample. As already noted, nine grantees expended no funds for OJT at all. 
Among the 14 remaining, at one extreme two grantees allocated 24% of their total 
funds to its OJT programs, and another allocated 17%. 7 In contrast, many others 



4 All three grantees were tribal grantees located in rural areas. 

5 Although the absence of employers was cited as the key factor, one of these grantees reported 
that, among the few employers that were accessible to participants, racism prevented those employers 
from hiring INA participants. The grantee noted that even the subsidy offered as a part of the OJT 
program was not a sufficient incentive to generate interest in the INA program among most local 
employers. 

6 Employer abuse may take the form of paying wages lower than market value, failing to provide 
meaningful training, etc. Employer abuse of OJT service activities has not been limited to the INA 
program, and has been documented among agencies and organizations offering OJT services elsewhere 
in JTPA. Technical assistance in developing high-quality relationships with employers, thereby 
diminishing the potential for employer abuse, may be of benefit to grantees concerned about this issue. 

7 One of these grantees had not offered OJTs at all for the previous several years. However, 
changes in the local economy spawned interest in job development that targeted particular industries. 
The grantee invested heavily in OJT programs as a primary vehicle to support this economic 
development strategy. 
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allocated only a percentage point or two of their total funds to support OJT activities. 
On average, however, sampled grantees who used this service activity spent 
approximately 6.2% of their funds for this purpose. Similarly, participants who were 
enrolled in OJT services represented a fairly small percentage of total participants 
among most of the grantees in our sample— generally no more than a percentage or 
two, but in the rare case reaching about 20% of the total. 

Among the 14 sampled grantees who offered OJT as a service option to INA 
participants, staff identified important benefits of this service in comparison to others 
offered by the program, including the: 

• Potential for long-term employment. Because OJT services are guided 
by formal written agreements (contracts) between participants, program 
staff, and employers, the potential for long-term employment is often 
greater for participants in OJT programs than for participants in other 
work-based activities. 

• Potential for “career pathing. ” Participants who integrate training and 
employment may be more likely to continue to train on-the-job, even if 
the INA program does not subsidize the training. This additional 
training enables participants to seek promotions and higher-wage job 
opportunities as they master new skills, ultimately improving their 
standard of living. In addition, some grantees target entry-level jobs or 
industries with high turnover and opportunities for advancement so that 
participants in these positions can gain access to new jobs as they master 
the skills required in their entry positions. 

• Potential for varied employment opportunities. Employers participating 
in the OJT program can be associated with the public or private sectors. 

As a result, the range of training and employment opportunities 
available to participants has the potential to be quite broad. 

• Potential for mentoring relationships. Because a formal training plan is 
required as a part of all OJT programs, employers generally ensure that 
a manager or long-term employee monitors the OJT participant during 
training. These individuals can “model” appropriate work-site behavior 
and assist OJT participants in learning company “culture,” in addition 
to specific training content. Such comprehensive learning better 
prepares participants for jobs and for eventual career advancement. 

• Potential for community benefit. Many grantees target Indian-owned 
businesses or private-sector firms that are seeking to build the 
infrastructure or meet the basic needs of Indian communities. This 
benefits Indian communities in at least two ways: (1) by subsidizing a 
portion of the costs associated with training and hiring while ensuring 
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that these firms have access to trained and appropriately skilled 
employees that will enable the firms to grow, and (2) by linking 
individuals to jobs, thereby supporting self-sufficiency and career 
advancement among community members. 

These benefits motivate grantees to continue to offer OJT services to their 
participants. 

There were also, however, clear challenges to grantees attempting to make OJT 
service options available to program participants. Some of these challenges are 
identical to those identified by the grantees that do not offer OJT services. As 
described above, these include funding challenges, low-levels of private-sector 
participation, low demand for OJT services among program participants, and the 
potential for abuse of OJT programs by employers. There were also some additional 
challenges that grantees providing OJT services identified, including: 

• Matching the skills of participants to the human resource needs of 
employers. Grantees in areas with little private sector development in 
their service areas or in areas dependent upon single industries indicated 
that the small number of employers or their lack of diversity made it 
difficult to effectively achieve this match. Grantees in more urban areas 
reported that, although nearby employers were actively hiring, their 
participants did not possess skills in demand. 

• Developing high-quality relationships with employers participating in 
OJT programs. Most grantees acknowledged the importance of 
developing relationships with employers to ensure that current OJT 
participants receive quality training and to maintain employers’ interests 
in OJT programs. In some cases, grantees had simply not been 
successful in developing such relationships. 

• Developing/securing approval for OJT contracts. Tribal grantees, in 
particular, reported that the frequently lengthy tribal approval process 
for OJT contracts made it difficult to maintain employer interest in the 
program. 

• Ensuring that meaningful training occurs as a part of OJT programs 
Grantees, both tribal and non-tribal, also reported that they were in need 
of training or technical assistance in developing operational work-based 
training plans and working with employers to ensure that such training 
actually occurs. 

• Assisting participants in transitioning to a full-time work-environment. 

Some grantees indicated that OJT participants could not successfuly 
make this transition, and dropped out of the program before their 
training program was complete or even just after they finished. These 
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grantees seek to develop ways to help their participants successfully 
navigate the transition to long-term employment. 

• Sustaining funding and internal support for the program. Several 
grantees indicated that their OJT programs were under threat as a result 
of budget cuts or overall financial strain. This pressure has reduced the 
number of OJT "slots” available in several sites, and impacted the 
quality of services provided in others. 

Despite the innovative strategies that some grantees have developed, which are 
discussed later in this chapter, these challenges prevent many grantees from making 
effective use of OJT, and prevent other grantees from offering OJT services at all. 

Grantees’ Objectives for OJT Programs 

Most grantees approached OJT programs with a healthy dose of skepticism. 
Respondents were well versed in the more controversial aspects of this service activity, 
including the lack of safeguards to ensure employers actually do hire participants in 
accordance with their contracts, and repeatedly cited these issues, among others, as 
barriers to the development of high-quality OJT programs. While such skepticism is 
not wholly undeserved, it has clearly prevented many grantees from fully integrating 
OJT programs into the menu of services they provide to participants. More 
importantly, because most grantees use the OJT program infrequently, many have not 
developed an overall strategy for the program in relation to other service activities, 
including identifying specific program objectives and target groups of employers and 
participants, and developing processes for follow-up during and after OJT programs are 
completed. 

In addition, because the program is not tremendously popular among JTPA 
program managers and supervisors who are most keenly aware of past instances of 
employer abuse, there is little incentive among front-line staff to fully develop these 
programs or attempt innovative approaches to OJT service activities. More often than 
not, grantees cited OJT programs as “too expensive,” even though many of these same 
grantees promote CSE and WEX as service activities, which can be as or more costly. 
Alternatively, grantees reported that OJTs were considered “more trouble than they’re 
worth,” despite their greater potential for long-term employment as an outcome in 
comparison to other work-based training activities. 
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Among the grantees that did offer OJT service activities to their participants, the 
following outcomes were most frequently cited as the primary objectives of the OJT 
program component: 

• Long-term placement for participants. Although there is an expectation 
of placement subsequent to the training component in all OJT programs, 
most grantees seek to ensure that the placements are long-term and of 
sufficient quality to provide for family self-sufficiency. 

• Employability for participants. In addition to employment, most 
grantees seek to ensure that OJT participants become employable— that 
participants are sufficiendy trained in an industry or in specific skill-sets 
to be able to attain employment after they master the initial job tasks, in 
the absence of an eventual offer of long-term employment by the 
participating OJT employer. 

• Supporting targeted economic development. As Indian communities 
seek to develop their infrastructures, many are targeting the 
development of specific industries. OJT can be an effective means of 
training a local workforce to support those industries— several grantees 
use OJT to support this. 

• Recruiting private sector employers. Because the OJT program offers 
services to employers— assistance in recruiting employees and a partial 
reimbursement for the costs of training— as well as participants, it can 
provide incentives for employers to participate in subsequent OJT 
programs or in other components of JTPA INA programs. 

Although these were the general objectives of the OJT component of the JTPA 
INA program, as cited by grantees, specific objectives varied, depending upon 
participants recruited and the types of OJT programs offered. 

Designing and Delivering OJT Services 

The OJT program is unique in that it serves two customers— the participants and 
the employers. While other JTPA service activities offer short-term training and work 
experience, such as WEX and CSE activities, the OJT program is guided by a formal 
contractual relationship in which employers become explicit customers of the JTPA 
program. As a result, designing OJT programs requires JTPA staff to skillfully 
negotiate a balance between the interests of individual participants and those of the 
employers, so that both sets of customers realize high-quality outcomes. 

Three key interrelated factors influenced the design and delivery of OJT services 
among the grantees in our sample. These included: (1) the overall approach to JTPA 
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services generally, and to the OJT program in particular, adopted by the grantees, (2) 
the local context in which JTPA programs operated, and (3) the adequacy of support- 
financial and otherwise— available to support JTPA participants generally, and OJT 
participants in particular. To varying degrees, each of these factors shaped the way 
OJT programs were designed and caused grantees in our sample to develop programs 
that exhibited considerable diversity within and across local sites. 

There were two dominant patterns with regard to designing OJT service activities 
among the grantees in our sample: (1) those arranging OJT slots with diverse 
employers, typically for work-ready participants, and (2) those arranging multiple OJTs 
to support targeted economic development among specific firms or industries. 

Grantees tended to use both approaches, either alternately (depending upon the 
local economy, environment, or context) or with different segments of the communities 
they served. However, most grantees’ overall program designs and services were 
dominated by one of these patterns. What follows is a discussion of grantees’ 
experiences in implementing each approach, including a description of the benefits and 
challenges they encountered as they sought to provide quality services to their 
communities. 

Approach #1: Providing OJTs with Diverse Employers 

Most grantees serving diverse participants had established “service tiers”— or 
menus of service options appropriate for participants requiring different levels of 
assistance from the program. Among the grantees that use this “tiered” approach to 
service delivery, OJT generally represented a service activity for participants who 
required little direct intervention or assistance, either because they had progressed in 
the JTPA program, or because of the skills and experience they possessed when they 
enrolled in the program. These grantees generally targeted particular groups or 
individuals for participation in their OJT programs, then identified employers who 
could benefit from the skills and experiences of these individuals, and developed OJT 
programs that would meet the needs of both sets of customers. 

The majority of grantees using this “tiered” approach to service delivery based 
their referrals to the OJT program on one or more of three sets of characteristics: 

• Significant work experience. Although there was not a specific work 
experience requirement for participation in OJT programs, most 
grantees sought evidence of participants’ basic workplace skills, such as 
reliability, appropriate workplace behavior and appearance (attire) , and 
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ability to follow directions or work with others. Participants who had 
held a job over time (more than six months with the same employer) 
were generally assumed to be competent in these areas. 

• Transferable Skills. Participants referred to OJT programs were not 
assumed to be seeking work in the same field or industry in which they 
possessed experience. However, participants who possessed skills that 
were easily transferable to other career areas were more likely to be 
referred to OJT programs than other participants were. For example, 
home caregivers— individuals caring for elders or children without 
pay— who were interested in careers as caregivers were frequently 
referred to OJT programs. 

• Specific (and informed) career interests and objectives. Some 
participants seek specific jobs or career opportunities through the JTPA 
program. These participants may have informal experience or skills, or 
they may just have strong interests. If their interests and expectations 
are informed, they too are likely to be referred to OJT programs. For 
example, a participant may have assisted in a particular vocation 
associated with a family business or profession, and seek to enter that 
career area. He or she would benefit from some formal training and 
work experience on the job. 

In addition to these three criteria, a high-school diploma or GED equivalency was also 
preferred for OJT participation among most grantees serving diverse participants. 

These factors were taken as evidence that a participant was job-ready, and such 
participants were more likely to be slated to receive OJT than others. 

Using Assessment in Designing OJT Services. In designing OJT programs for 
participants initially determined to be job-ready, most grantees emphasized skills, 
experiences, and training needs over general career interests in assessing participants’ 
ability to participate in a particular OJT occupation. Job-ready participants were not 
necessarily required to complete the full battery of assessments recommended to 
participants requiring more comprehensive assistance. Although this approach is 
clearly a more efficient use of staff and participant time and resources, it is crucial that 
the initial assessment is accurate— and this was not always the case. For example, the 
grantees’ informal assessment of participant experience sometimes caused participants 
to be placed in jobs that they may have been qualified to do, but were not interested in 
doing. This can lead to dissatisfaction for both employers and participants in the short- 
term, and compromise the grantees’ program quality over time. 
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Most grantees did require that participants initially deemed to be job-ready take 
the basic reading and math skills assessments, because these skills were required for 
most OJT positions. Many grantees set a kind of informal threshold that participants 
were required to achieve prior to enrolling in an OJT program. If skill-deficits were 
revealed during these assessments, staff generally made alternative referrals— to basic 
skills classroom training or other skill-building programs . This threshold served as a 
check to ensure that individuals with experience or interest also possessed the skills 
required for them to succeed on the job. Alternatively, it can also help case managers 
to work with employers to design programs that were appropriately challenging to 
participants. For example, one participant with a diploma and two years of college 
math was still required to take the basic math skills assessment prior to her accounting 
OJT program— not surprisingly, she received a nearly perfect score. This gave her case 
manager and future employer great confidence in her abilities, and prompted the two to 
work together on a more intensive training plan than had originally been proposed. 

Developing Appropriate Service Mix. Grantees designing diverse OJT 
positions for work- ready participants tended to offer OJT as a stand-alone activity, or 
sometimes pair it with classroom training. These grantees perceived OJT programs as 
activities appropriate for participants who “just need a little help”— they might be 
skilled but unable to locate appropriate employment, experienced but without a recent 
work history, or motivated and interested but lacking experience. The OJT program 
can offer structure, training, and paid work-experience, satisfying many of the needs of 
these participant groups. Alternatively, OJT served as a school-to-work activity for 
participants who completed vocational or occupational training or their high school 
equivalency, and sought to transition into full-time employment. 

Developing Appropriate Service Length/Intensity. Grantees that arranged OJT 
positions with diverse employers for job-ready participants also tended to emphasize the 
development of OJT programs that were customized to meet the needs of individual 
participants. As discussed above, one way this occurred was through the combining 
and sequencing of multiple service activities and supports. 

Another strategy was varying the length or total hours of OJT programs to best 
meet the needs of the participant, employer, and grantee, within overall limits set by 
the grantees and federal regulations. For example, although the legislation limits OJTs 
to 6 months, a few grantees placed their own (formal or informal) caps on the number 
of days/hours participants may engage in OJT programs. One grantee, for example, 



limited OJT contracts to 13 weeks— half of the allowable program length under federal 
JTPA regulations. Alternatively, one grantee placed a monetary cap of $1 ,300 on OJT 
contracts, regardless of the number of hours worked. These strategies permit grantees 
to offer OJT as a service option to more JTPA participants. However, they also reduce 
the capacity of the program to market the service to employers because the incentives 
for employer participation, their wage reimbursements, are not as significant. 

Within these limits, most grantees worked with individual employers to identify a 
period of time after which participants were expected to perform at the level of other 
employees, rather than as trainees. For example, if the expected training time for an 
experienced new employee working for a particular employer was two weeks, then a 
JTPA participant who might have formal training but no experience might be working 
on an OJT contract for 2-3 times longer. 

Grantees have also worked with employers and participants to develop OJT 
contracts around particular competencies. For example, a grantee might agree to 
support an OJT contract for no less that 8 but no more than 12 weeks, by which time 
the participant must be able to demonstrate some minimum level of competence in 
required skill areas. Grantees have also worked with employers and participants to 
develop programs that demand two-, four-, or six-hour workdays rather than the 
standard eight-hour days, so that the participant may engage in other service activities 
(e.g. classroom training) or meet child-care or other obligations. For example, one 
grantee designed a four-month OJT in which a participant worked in the 
accounting/billing department of an Indian-owned firm, and enrolled in related 
coursework for two-hours per week during that time. At the conclusion of the OJT, 
the participant planned to continue working with the firm, taking on increasing 
responsibility, and enroll in additional coursework leading to a degree in accounting. 

The ability to adapt OJT programs to meet the specific needs of employers and 
participants is critical for grantees providing services to participants with diverse skills 
and experiences. 

Developing Training Plans. Grantees are required to develop a training plan for 
all JTPA OJT contracts. At a minimum, the plan should include: (1) an oudine of the 
length and nature of the training and the specific skills to be taught, (2) a description of 
the job for which the participant is being trained, (3) a plan for ensuring that 
participants progress throughout their OJT program, and (4) a reimbursement schedule 
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or identification of the maximum subsidy the employer can receive as a part of the 
contract. Nonetheless, because grantees that emphasized OJTs with diverse employers 
designed primarily individualized OJTs, there was great variation in the content, 
duration, and quality of the training plans. 

Most grantees we visited did ensure that there was a written document outlining 
planned training content and objectives for each participant. However, the level of 
detail describing how the training would actually take place, and identifying the 
supervisors’ strategies to ensure that skills were learned, varied widely. Where training 
plans were well developed, they could be used to help participants and employers 
benchmark participant progress and ensure that the expectations of employers, 
participants, and case managers were being met. They also served as a “check” on 
program coherence by ensuring that new skills participants were required to learn 
related directly to the positions for which they were being trained. Perhaps most 
importantly, training plans helped participants seeking long-term careers begin planning 
for them. When participants and their case managers used their training plans to 
monitor progress, participants were better able to articulate the skills they were learning 
as a part of their OJT programs and understand how those skills might be applied in 
other employment settings or in other future activities. 

For example, we interviewed one participant who was enrolled in an OJT 
program within a tribal enterprise (small-business). She and her case manager, 
together with her current supervisor, developed a training plan prior to her work with 
the tribal enterprise. She was very familiar with her training plan and was able to 
identify specific skills she had acquired during her OJT, as well as “professional 
development opportunities ” 8 she had requested and completed. Another enterprising 
young participant with whom we spoke also worked closely with her case manager and 
supervisor to develop her OJT training plan. She was placed in an office environment 
and learned about numerous professional development opportunities in which she was 
interested in participating. When she requested support for these opportunities, her 
supervisor asked her to develop a proposal explaining how these opportunities were 



8 The participant used this phrase to describe two out-of-town seminars she had attended which, 
although they were not part of her original training plan, she had suggested to her supervisor might be 
beneficial— he agreed and she attended the seminars. 



VII-14 

188 



consistent with her original training plan. She drafted a letter and was able to take 
advantage of two such opportunities. 

Another grantee described an instance in which a former participant was able to 
demonstrate skills achievement with his training plan, which was signed by the 
participant, the case manager, and the employer at the conclusion of his OJT program, 
and earn credit toward his Associates Degree. 

The majority of the participants with whom we spoke who actively participated in 
the development and ongoing revision of their training plans expressed greater clarity 
with regard to not only their training programs, but their comprehensive personal and 
career goals than did participants in OJT programs who were not as familiar with their 
training plans. 

Finally, when training plans were used to guide OJT program implementation, a 
clearer break existed between the training program and the subsequent job— whether the 
job was with the OJT employer as intended, or with another employer. This break can 
enable participants to work with their supervisors and case managers to renegotiate 
their job descriptions and their wages. Since OJT programs are training activities, the 
wages tend to be run below current market value— the break between training and 
permanent employment can provide an opportunity for participants to renegotiate their 
hourly rate or salary. 

One participant, for example, was able to document her progress in achieving her 
OJT program goals and use this in clarifying her new roles and responsibilities. 

Because she could demonstrate that she met her training objectives ahead of schedule, 
she successfully transitioned into a position with considerably more prestige, 
responsibility, and remuneration. 

These anecdotal cases illustrate some of the potential benefits of developing 
effective and useful training plans. Although shorter OJT training programs may not 
merit lengthy, sophisticated plans, a simple process of identifying the skills to be 
learned and developing a process to ensure that learning takes place can have clear 
benefits for participants, employers, and program staff. 

In contrast, during our random case file reviews and in examining the case files 
of the participants with whom we met, we found instances of insufficient— or non- 
existent — training plans. In other cases, training plans were developed without the 
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knowledge and input of the participant. For example, in some cases, an employer’s job 
description served as the training plan. If these job descriptions identified specific 
skills to be developed and included some mention of specific means of developing 
them, they may have indeed served as adequate plans. However, some of the job 
descriptions included only the job title, the employer’s name, and one or two sentences 
describing the position. Such descriptions were not effective training plans. 

Alternatively, in some cases training plans were developed, but were not specific 
enough to be used in monitoring participant progress. For example, one training plan 
identified "learning office procedures” as one of three training objectives . 9 Although 
this is a reasonable overall objective for an OJT program, it is "open-ended” and does 
not enable participants to assess their own progress or skills gains— presumably the 
participant will always be learning office procedures as new practices are adopted or 
procedures developed. Moreover, we also learned about conflicts that arose when the 
participant thought they had mastered a general skill area, but the supervisor disagreed. 
Training plans that identify specific skills, rather than more general categories in which 
learning takes place, may serve to prevent such misunderstandings. 

Finally, in some cases training plans were developed either by the case manager 
alone, or the case manager and the OJT supervisor collectively, without the input of the 
OJT participant. Although the case managers and supervisors, as professionals, are 
amply qualified to write quality training plans, when participants are not involved in 
the process their understanding of the OJT program and activities is undermined and 
their ability to take an active role in their own training is compromised. In 
interviewing participants, we learned of instances in which training plans were 
developed that were inconsistent with participants’ expectations. This created tension 
between participants and their supervisors, and compromised the relationship between 
employers and JTPA program staff. 

Developing useful training plans for individual OJT participants represents a 
challenge for grantees. We found great variation in the types and quality of training 
plans not only among grantees but also among different agencies associated with the 
same grantee, and even among different OJT programs developed by the same agency, 



9 The other objectives included “learning about the business” and "learning to work with 
others.” 
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but by different agency staff. Developing individualized training plans for OJT 
participants is a challenging but crucial part of providing high-quality OJT experiences 
for participants that lead to long-term employment and positive individual and 
community outcomes. 

Support, Supervision, and Oversight of OJT Activities. Like the variation in 
type and quality of training plans, we also observed significant variation in the 
frequency and quality of ongoing monitoring activities among grantees whose program 
designs emphasized services to participants with diverse skills and experiences. Most 
of these grantees indicated that they encouraged case managers to maintain at least 
monthly contact with JTPA participants to assess participant progress, provide 
necessary support, and ensure that the plans driving their work or training programs 
remain useful and accurate. Our interviews with case managers and participants and 
our review of participant files indicated that such contact was less likely to be 
maintained when participants worked or trained “off-site” 10 — and OJT participants in 
particular were more likely to be placed “off-site” than participants in other service 
activities. As a result, the frequency and intensity of contact between participants and 
case managers or case managers and supervisors was largely a function of case 
managers’ interest, diligence, and time availability. 

At one extreme case managers had almost daily contact with OJT participants. 
This was common where participants were placed in the Employment and Training 
Department or other agencies or nearby offices, or when the case managers and 
participants lived near one another. At the opposite extreme, we spoke with case 
managers who do not typically initiate contact with the OJT participants once they are 
placed, although they will respond if a problem arises. The few case managers who 
adopted this approach reported that once participants were placed, they did not want to 
interfere by checking up on them because they feared that employers would perceive 
such efforts as intrusive . 11 More common however, was weekly to monthly contact 
between participants and case managers, although contact between case managers and 



10 In some cases this meant on the reservation but in another city or town, in some cases it meant 
off of the reservation entirely, and in some cases it meant simply “far enough” so that case managers 
would be unlikely to run into participants locally. 

11 None of the employers with whom we spoke indicated that their contact with the agencies or 
departments coordinating OJT programs had been a burden, and some indicated that they would have 
preferred wore contact with program staff. 
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participant supervisors was much less frequent and tended to be driven by specific 
issues or problems rather than processes or protocols. 

The quality of interaction between case managers and participants and case 
managers and supervisors also varied considerably. Interviews with participants, 
employers, and case managers, together with case file reviews, revealed inconsistencies 
in the objectives of such contact as perceived by all of the key stakeholders, as well as 
differences in the information shared during contact. Not surprisingly, contact was 
most often initiated by the case managers, rather than the participants or the OJT work- 
site supervisors. Case managers approached these interactions in a variety of ways. 
Case managers who adopted a more hands-on approach to monitoring OJT programs 
tended to contact participants and supervisors separately (but at the same time intervals) 
and ask similar questions to each respondent at each interval. Common questions 
addressed participants' progress, job satisfaction, unexpected challenges or difficulties, 
milestones, as well as participants’ overall physical, mental, and emotional health. The 
case mangers then compared the responses to ensure that they were reasonably 
consistent, and offered congratulations, words of encouragement or intervention 
services as needed. 

Alternatively, some case managers contacted participants on a regular basis 
simply to ensure that they were still working with the same employer. This level of 
contact produced little more than a verification of employment status for a case file. 
However, most case managers take a more moderate approach than either of these two 
extremes, contacting participants more frequently than their supervisors and 
maintaining a willingness to provide additional support or intervene as needed. 

For these case managers, the challenge lies in knowing when intervention or extra 
support is necessary or advised. This requires both a quality training plan and ongoing 
monitoring of participants— neither alone is sufficient. For example, in one case 
participants in an OJT program that was designed to train them in basic office skills 
were increasingly asked to perform heavy lifting and other duties associated with the 
day-labor side of the business — where participants tend to be higher paid— to the 
exclusion of their formal training in office procedures. From the perspective of the 
employer, this was perfectly legitimate because he was both using personnel resources 
more effectively (when there was no office work, participants were reassigned as 
needed) and exposing participants to a broader menu of skills than just those associated 
with office procedures. Since the participants liked their boss and their jobs, they 
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responded positively to case managers’ inquiries. If a quality training plan were 
guiding the OJT program and the questions asked by the case manager, the case 
manager may have been better able to assess the situation and identify the problem. 
Instead, the participants continued on in their OJTs where they were ultimately trained 
in a random mix of tasks that lacked coherence. This lack of formality prevented the 
participants from assessing their own progress as they moved toward specific 
objectives. They were ultimately well prepared to work in their current setting, but the 
absence of more specific skills training may ultimately prevent them from moving into 
positions of more responsibility, or those that better suit their personal and professional 
needs. 

Another benefit of developing effective training plans and establishing a good 
rapport with participants and supervisors is that it serves to prevent much of the abuse 
that many grantees fear. A common form of employer abuse with regard to OJT 
programs has been the refusal of employers to hire OJT participants at the conclusion 
of their training. However, a training plan that focuses on teaching participants the 
skills most in demand at the firm or organization, together with sufficient monitoring 
and support to ensure that participants master those skills, can serve to make 
participants valued staff members by the conclusion of the OJT and reduce the 
likelihood that the employer will seek to terminate participants’ employment. 

Employer abuse can also take the form of inattention to the training plans 
developed as a part of participants’ OJT programs. In such cases, employers might 
hire an OJT participant for a specific position or area of expertise. However, rather 
than following the training plan and ensuring that the participant learns what is required 
for the permanent position, the employer might place the person wherever there is a 
shortage of help. The participant functions like a temporary worker. This strategy 
benefits the employer by enabling him or her to collect a training subsidy without 
providing much training. The participant benefits from the wage, but does not receive 
training consistent with the original OJT design. Again, in these cases, a quality 
training plan, combined with an appropriate level of interaction between the case 
manager and participant, and the employer and case manager, can prevent such 
intentional misuse of the OJT program or unintentional misunderstandings of a similar 
nature. 

Finally, another form of employer abuse that reportedly impacted grantees’ 
willingness to provide OJT programs for participants was the potential for employers to 
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offer a lower than market- value wage to OJT participants. Again, a quality training 
plan, together with an appropriate level of staff involvement in the design and 
implementation of the OJT program, can ensure that the employer, the participant, and 
the grantee are satisfied with the program and its outcomes. 

Serving participants with diverse skills and experiences offers opportunities for 
high-quality matching of employers and participants, marketing OJT to a wide variety 
of employers, and promoting career-advancement and self-sufficiency among 
participants. However, high-quality OJT programs can absorb administrative time. 
Developing protocols and processes and ensuring that staff are appropriately trained can 
streamline the process and help grantees meet the specific needs of individual 
participants and employers, as well as the needs of the larger Indian community. 

Approach #2: Arranging OJTs in Targeted Industries 

Grantees also designed OJT programs to support a focused local strategy of 
economic development or otherwise limited targeting of employers or industries. For 
example: 



• Local economic development strategy. In some areas served by tribal 
and non-tribal grantees, employment and training providers had 
established relationships with economic development entities and 
employers to promote particular economic development strategies, such 
as targeted-industry development. As a result, the employment and 
training providers, including the JTPA entities, tended to develop 
programs and services that would support this development by 
recruiting and training individuals with specific skills and aptitudes. 

• Limited targeting of employers or participants. Some grantees have 
(explicitly or informally) identified either specific employers (e.g. , the 
largest employers in town) or specific types of employers (e.g., those 
within a target industry or those lacking in sophisticated human resource 
supports) , and then recruited participants and developed OJTs to meet 
the needs of those employers. 

Typically, multiple OJTs were developed for these positions, either simultaneously or 
consecutively. The experiences of grantees using this approach to design and deliver 
OJT services are discussed in this section. 

Recruiting Participants. Grantees arranging OJTs with targeted employers or 
industries used several different strategies to recruit individuals for participation in their 
OJT programs. These strategies included: (1) identifying individuals with skills in 
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demand among local employers, (2) identifying individuals with appropriate interests or 
aptitudes, (3) identifying individuals with an intense need for program assistance to 
achieve employment and employability, and (4) identifying individuals with jobs or 
good potential for employment who would benefit from additional assistance under 
JTPA. Each of these is addressed below. 

Some grantees engaged in supporting targeted economic development worked 
with local private-sector employers to identify the skills required for available jobs. 
Grantee staff then referred JTPA participants who exhibited those skills, or at least the 
potential to develop those skills, to the OJT program. This did not necessarily mean 
that the participants were job ready or had attained a high -school diploma or GED, but 
that they exhibited the willingness or potential to learn the skills necessary to perform 
on the job. Employers benefited through the subsidy to support customized training 
and the development of a current and future workforce to the extent that OJT 
participants could advance in their careers. 

Grantees also recruited participants who demonstrated an aptitude or interest in 
careers with employers who had available jobs. These individuals did not necessarily 
possess the skills required by local employers when they enrolled in the JTPA program, 
but they may have demonstrated a good work ethic or some general knowledge of 
employers’ industries. 

Some grantees targeted the individuals most in-need of services for participation 
in the JTPA program. In such cases, individuals referred to the OJT program were 
generally expected to learn general work skills in addition to skills associated with their 
particular positions. In such cases, there was no expectation that the individuals would 
remain in the same career area. However, the OJT and subsequent work experience 
was understood as crucial in enabling them to find subsequent work. 

Finally, some grantees sought to support training among their working or work- 
ready participants for whom training might otherwise not be available. In some cases, 
these individuals were referred to the JTPA program after they had already been hired 
(reverse referral), and, in some cases, participants were referred as a condition of their 
hire. Grantees who recruited OJT participants through these methods reported that this 
training helped participants to advance and helped establish a good rapport between the 
grantee and participating employers. 
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Using Assessment in Designing Services. We found that the role of assessment 
to design OJT services when participants exhibited similar skills and experiences varied 
enormously from grantee to grantee, from agency to agency within the same grantee, 
and from employer to employer (many grantees use employers assessments to inform 
their selection of OJT participants). In general, there was more emphasis on specific 
skills assessment among grantees using this approach than those serving a more diverse 
participant base; however, this may have been because these grantees tended to recruit 
multiple participants (in the case of targeted industry development) rather than one at a 
time. As a result, skills assessments, either the grantees’ or the employers’, were often 
used as “screens,” permitting grantees to refer participants who did not pass to other 
JTPA activities. On the other hand, however, when participants were referred to the 
JTPA program by their employers, there was rarely an assessment conducted by the 
grantee at all because it was assumed that the employer was generally satisfied with 
participants’ skills. 

One challenge that grantees using this approach faced in assessing participants’ 
skills was ensuring that their needs and interests, in addition to those of the employers, 
were taken into account in developing OJT programs. Because grantees used OJTs in 
supporting targeted economic development in their local communities, they were 
frequently placing more than one OJT participant with an employer at the same time. 
Because the OJT program is often perceived as an effective way to market the entire 
JTPA program to employers, grantee staff reported feeling pressure to make the needs 
of employers a priority, because there were typically so few of them (and so few jobs) 
in their local service areas. This pressure resulted in staff placing participants in OJTs 
who may not have been interested in or prepared for these programs. Grantees face the 
challenge of ensuring that participants understand the nature of the job and are prepared 
for work before they are enrolled in OJT programs. 

Developing an Appropriate Service Mix. Like grantees using the first 
approach— targeting work-ready participants for diverse OJT positions— grantees 
targeting specific industries or employers also tended to design OJT programs as stand- 
alone activities or paired with classroom training. These grantees perceived the OJT 
training program, particularly when it was paired with vocational training, as a means 
of supporting the development of a local workforce skilled in areas in demand among 
local employers. For example, one grantee providing services in an area dominated by 
the tourist industry worked with a local training provider to develop curricula in 
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hospitality and customer service. The grantee then designed OJT programs for 
participants interested in this industry that combined a component of CRT with their 
paid OJT work. Not only were participants prepared for their entry-level jobs, but the 
training they received prepared them for career paths in the industry. 

Moreover, for grantees providing OJT services to employers in areas dominated 
by a few industry clusters— timber, tobacco, manufacturing, etc. — the OJT program can 
serve as a means of recruiting additional employers if the grantee was able to establish 
an effective track- record in placing qualified candidates in these industries. The OJT 
program also offers grantees (or service providers with whom grantees coordinate) the 
opportunity to build their capacity to provide customized training to meet industry or 
employer needs. For example, another grantee serving an area with very few 
employment opportunities developed an OJT program for a new employer in the 
service industry. The program enabled the employer to hire a qualified workforce for 
the employer’s grand opening. When another (service-oriented) business located next- 
door, the newer employer was referred to the JTPA program by the original employer. 
While few grantees had identified this practice as a specific employer recruiting 
strategy, the practice appears promising. 

Developing Appropriate Service Length/Intensity. Grantees using this 
approach to designing OJT services also varied the length and training time associated 
with OJT programs. However, cost, rather that the needs of individual participants, 
was most frequently offered as a rationale for placing these limits on programs. These 
grantees found that limiting the total reimbursement amount, or total number of hours 
in which participants could be enrolled in OJT programs, was effective in reducing the 
overall cost associated with the program and transitioning participants to unsubsidized 
work more quickly than the six-month allowable time frame associated with the 
program. For example, one grantee limited OJT contracts to $1,300 (the average 
length was about six-weeks) , combining the OJT with a informal on-site group 
instruction, which enabled the site to offer OJT services to more employers and 
participants. Alternatively, some grantees limited the number of participants they 
would enroll in an OJT program on an annual basis. Within these limits, there was a 
modest effort to vary training contracts to participants’ needs. 

OJT Training Plans/Content. Grantees that emphasized OJT contracts designed 
to support industry development or meet the specific needs of employers were also 
required to develop training plans for participants as a part of the program, consistent 
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with the requirements for OJT in JTPA. Since most grantees using an employer- 
focused approach worked with core industries or groups of employers on a regular 
basis and developed programs for the same types of positions, the OJT programs 
developed in this context were generally well-established and institutionalized. When 
participating employers identified a hiring need that could be addressed through an 
OJT, their JTPA liaison would identify an appropriate candidate, and then, rather than 
developing a training plan from scratch, the staff person was likely to base the new 
training plan on a previous one developed for another participant— some grantees 
developed “templates” for OJT contracts and training plans, and simply inserted 
participants names in the blank spaces. Participants and staff involved in these types of 
programs typically spoke of them as if they existed independently of the participants— 
participants, in particular, had a sense that there was a pre-existing program in which 
they were placed, not that the program was designed around individual participants. 
New participants coming into OJT programs where grantee and employer relationships 
were institutionalized experienced a number of benefits. These included: (1) regular 
and ongoing communication between grantee staff and employers (reducing the chance 
of misunderstandings or problems between these two stakeholders) , (2) employer 
experience in providing training using the OJT program (improving the likelihood that 
the training would be meaningful and ongoing) , and (3) the potential for mentors at the 
work-site who had also gained employment by participating in the OJT program. 

Although relying on current relationships or using prior training plans to design 
OJT services for participants is an efficient use of staff time and program resources, 
grantees using this strategy face the challenge of balancing the needs of the program, 
the participant, and the employers to ensure that the needs of all of the key stakeholders 
were addressed in the most resource-efficient and effective way. For example, several 
grantees had been working with employers for so long that the employers either trusted 
the grantees to develop quality plans, or encouraged them to use training plans 
developed for other participants who had completed similar programs. In some cases, 
this is appropriate — it is a good use of program resources and meets the needs of the 
employer and the participant. However, in other cases, it resulted in no formal training 
plan at all— participants were not clear on the skills they intended to learn or what was 
expected of them on-the-job, and when problems arose, their files contained no 
documentation that could assist the grantee staff, participants and employers in revising 
their programs. Challenges that might have otherwise been turned into learning 
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experiences remained problems and compromised relationships between the key 
stakeholders. 

OJT Programs for Multiple Participants. Some grantees providing OJT 
services targeting to specific industries and employers developed OJT programs that 
were designed for groups of participants rather than individuals. For example, several 
grantees worked with employers in seasonal industries— such as construction or 
tourism— that tended to hire en masse. Other grantees targeted new or expanding 
employers that also planned to hire more than one participant. Still other grantees 
targeted specific groups of participants— women with experience caring for children, 
for example— and worked with multiple firms in the same industry to place them. 

When grantees worked with multiple firms in a single industry or with employers 
seeking to hire groups of new workers, they tended to develop OJT programs designed 
to serve multiple participants simultaneously. 

For example, one grantee worked with a new employer to meet the majority of 
the incoming firm’s human resource needs. Since the firm required about 20 similarly 
qualified individuals for a single job category, one program was designed for these 20 
individuals, and a single contract was signed between the employer and the tribe, with 
each individual participant signing and receiving her or his own copy of the contract. 
Another grantee worked with a community college to design a curriculum that would 
meet the needs of the tourism industry. In this case, individual contracts were 
developed, and the on-the-job components of the programs were customized, but 
participants received identical instruction in the basic skills required as a part of the 
industry. 

Grantees reported a number of benefits resulting from this strategy, including: (1) 
the increased likelihood of comparable training for participants, (2) increased 
involvement among local employers in developing OJT and other JTPA programs, and 
(3) increased efficiency in use of program resources. Each of these is addressed below. 

• Comparable training for participants. When grantees, contracted 
service providers, or employers develop training for groups rather than 
individual participants, they tend to offer a more formal and 
institutionalized training program than they would provide for a single 
participant. This ensured that participants were comparably qualified to 
assume the jobs for which the training was developed and made the 
transition to work easier on participants and employers. 
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• Involvement of employers in developing programs. Employers hiring 
groups of new workers had a greater stake in the hiring and training 
process than they would have if they had been hiring only a single 
participant. As a result, these employers tended to be very involved in 
the development of OJT contracts and the provision of training. 

Grantees reported that employer involvement tended to increase the 
overall quality of the OJT experience and develop the capacity of the 
grantee to assist employers in meeting future hiring goals. 

• Efficient use of program resources. Grantees reported that, although 
group-based OJT programs were labor-intensive up-front (developing a 
training plan that meets employer, providers, grantee, and participants’ 
needs) , they were an efficient use of program resources because 
grantees did not have to develop discrete and unrelated contracts and 
training plans for each individual served, and because of the anticipated 
program outcomes —multiple participants placed in living- wage jobs. 

Grantees also noted some of the challenges they face in developing group-based 
OJT programs and training plans for their participants. These challenges included: (1) 
the tendency for programs to become “formulaic” and inflexible, (2) the tendency for 
grantees to rely on a limited number of employers with whom relationships have 
become institutionalized, and (3) the tendency for grantees to rely on communication 
with employers, and not participants, to monitor the performance of OJT programs. 
Each of these is addressed below. 

• Formulaic programs. Some grantees reported that they relied heavily 
on previous contracts and training plans to develop new contracts and 
training plans with the same employers or with new employers in the 
same industry. 12 While this is cost-savings, steps must be taken to 
ensure that these documents are appropriate and useful, lest grantees 
compromise the experience for the participants or employers. 

• Limited number of employers. Some grantees worked to establish high- 
quality relationships with a limited number of employers served by their 
programs. While these employers served as reliable customers, some 
grantees were almost entirely dependent on a few employers or a single 
industry to provide jobs for their program participants. This narrow 
targeting reduced the choices for participants who were interested in 
other career opportunities but who were nonetheless encouraged to 
pursue jobs with the core group of employers. 



12 We saw one instance wherein a contract was developed on the basis of a previous contract 
with a different employer, but the employer’s name had not been changed throughout the document— 
the contract included the name of an employer that was not involved in the OJT program. 
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• Over-reliance on employers to assess program. When the employer was 
providing training for multiple participants, grantee staff tended to rely 
on the employer to assess participant progress and performance, rather 
than contacting the individual participants. Unfortunately, however, 
employer assessments were not always consistent with those of 
participants . 13 

Grantees that developed group-based OJT programs reported that they can be 
very effective tools for ensuring that JTPA participants benefit from new or expanding 
economic or industry development. 

Support, Supervision, and Oversight of OJT Activities. In general, grantees 
(or agencies or staff) that developed OJT programs for groups rather than for 
individuals, or used OJT as a strategy for supporting targeted industry development, 
had less frequent contact with their participants than grantees that focused on the 
development of individual OJT programs. Grantees cited several reasons for this. 

First, participants were rarely placed in employment situations that would make 
frequent contact with their case manager probable. For example, group-based OJT 
programs were almost never developed for tribal departments or agencies, because such 
agencies rarely had a need for multiple new employees for a single or even a few job 
categories. As a result, case managers were unlikely to “run into” participants in the 
same way as was common when participants were hired, via OJT, by tribal or grantee 
agencies or organizations on an individual basis. Although some participants did not 
require additional assistance or follow-up contact from their case managers after 
placement in their OJT programs, others would have benefited from a more formal 
follow-up approach. 

Second, participants working in group situations tended to rely on each other for 
support rather than solely on their case managers, and they thereby initiated contact 
with the case managers less frequently than participants in individual OJT slots. When 
OJT participants trained and began new jobs in groups, they tended to develop mutually 
supportive relationships among themselves rather than relying on their case managers. 
Moreover, larger group-based OJT program tend to be more institutionalized— the 
employer may have strategies, such as mentoring or frequent staff or team meetings, to 



13 In reviewing participant files, we saw evidence of program staff contacting employers (but not 
the participants) and assessing the participants’ progress on that basis. 
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prevent conflicts or address issues as they emerge. This reduces the frequency of 
participant-induced contact with case managers. 

Finally, case managers tended to use the employer as the primary point of contact 
for participants completing group-based OJT. Because the employer representative was 
the single point of contact for all of the OJT participants working at that site, case 
managers were able to generate a great deal of information in one simple phone call. 
However, contact with the employer alone is insufficient. For example, one grantee 
reported that an OJT participant who won rave reviews from her supervisors quit 
unexpectedly. Later the case manager discovered that the individual had transportation 
challenges that could have been resolved if she had made them known to her case 
manager. The case manager had been in contact with the employer, but not with the 
participant, and did not anticipate the problem. A review of case files among all of the 
grantees we visited revealed that frequent contact with employers participating in OJT 
programs was not matched by corresponding contact with individual OJT participants, 
particularly in group-based programs. 

Like grantees that emphasized the development of individual OJTs, the quality of 
interaction between grantee staff and participants in group-based OJT programs varied 
widely. Interviews with grantee staff, participants, employers, and reviews of case 
files revealed that, in some instances, this interaction was thorough and meaningful, 
while in other cases, it was only cursory. In some cases, quality of interaction was 
correlated with frequency of contact. For example, some case files revealed that case 
managers contacted participants often and solicited important information each time. In 
other cases, interaction that was infrequent nonetheless yielded significant and 
important information. 

In general, the grantee staff conducting follow-up on participants in group-based 
OJT programs had an advantage, because the participants’ training plans tended to be 
well-developed and institutionalized as compared to most individual OJT training plans. 
These training plans often served as a protocols for guiding case-manager and 
participant follow-up discussions— case managers were able to ask participants about 
their general satisfaction and progress, but also address specific competencies in an 
effort to ascertain whether participants were meeting their training goals. 

Compared to the first approach (developing OJTs for individual slots) , developing 
group-based or targeted OJTs also did not appear to pose as much of a threat of 
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employer abuse. At least, grantee staff rarely mentioned the issue. This suggests that 
group-based OJTs offer some protection from employer abuse because they tend to 
emphasize employer involvement in program design, leading to more customized 
training, better prepared employees, and greater satisfaction on the part of employers 
and participants. Moreover, the numbers of participants the JTPA programs can 
access, combined with the programs’ ability to put resources into training, give the 
JTPA program some leverage in negotiating wages for trainees and determining an 
appropriate length of training— these factors also diminish the potential for employer 
abuse. 

This second approach offers grantees the ability to closely link employment and 
training program development to local economic development, thereby promoting dual- 
track outcomes: a well-prepared labor force and employers offering living- wage jobs. 
The challenge lies in ensuring that the program serves all community members, and not 
just those interested in a particular industry or employer. Because particular employers 
are likely to present themselves as a reliable source of employment opportunities, 
grantees may be eager to meet these employers’ needs by identifying appropriately 
skilled individuals. However, Indian communities, like all communities, maintain 
diverse membership and requires diverse economies in order to thrive. Grantees can 
use their programs not only to support successful and growing employers or industries, 
and their labor forces, but also to support a wide array of small business thereby 
serving a more diverse group of employers and individuals within their communities. 

Worksites Associated with OJT Programs 

Unlike the WEX and CSE programs, participants in the OJT programs, 
regardless of the grantees’ design approach, were much more likely to be placed with 
private-sector employers or tribal-owned enterprises than in Tribal agencies or 
departments. Although the employers varied in size and industry across the grantees in 
our sample, they fell primarily into three different categories: (1) tribal enterprises, (2) 
Indian-owned firms, and (3) other employers. 

Several grantees used their OJT programs to support the development of tribal 
enterprises as a target industry. In some cases, grantees supported a single tribal 
enterprise, such as tourism or gaming. For example, one grantee used its OJT program 
to train participants for positions in the tribe’s gaming facility. The OJT program 
trained participants in both the skills they required to perform their jobs and in 
communication skills and tribal history— because the participants were placed in an 




VII-29 



203 



industry supported by tourists, they required a solid knowledge of the tribe’s culture 
and tradition. Another grantee worked with a training provider to develop a course 
designed to train participants for working in the tourism industry more generally. 

Some of these participants were enrolled in OJT programs with a tourism information 
center, and were co-enrolled in a course where they trained both in skills specific to 
that industry and skills transferable across other industries (e.g. accounting skills). 

Other grantees used OJTs to support the development of a variety of tribal 
enterprises, rather than a single industry. For example, one grantee developed OJT 
programs in two small, tribally-owned retail facilities, and an economic development 
agency. Although only one participant was placed in each establishment, the 
placements clearly benefited the employers, the participants, and the communities 
because they addressed so many needs simultaneously— participants were placed in 
living- wage jobs with opportunities for advancement, employers were able to hire 
trained, enthusiastic employees, and communities witnessed improved capacity (and in 
some cases, performance) among their local businesses. Examples of other tribal 
enterprises that provided training opportunities of OJT participants include child-care 
facilities, public services, and manufacturing. 

Grantees also targeted Indian-owned firms as potential recipients of JTPA 
services, and OJTs in particular. This was especially true among non-tribal grantees 
located in urban areas. These grantees reported that targeting Indian-owned firms 
fulfilled two objectives consistent with their overall program goals (1) supporting the 
growth of Indian -owned firms, which were generally small and appreciative of 
assistance in meeting their human-resource needs, and (2) offering participants 
opportunities for a work-life influenced by Indian culture (even if a different Indian 
culture from their own). Moreover, grantees also reported that many Indian business 
owners sought to “give back” to the Indian community— participating in the JTPA 
program via OJT was a good way to accomplish this. 

Finally, some grantees did not work with specific groups or types of employers, 
but attempted to develop OJT programs when they could create a good match between 
the employers’ needs and the participants’ skills. In these cases, employers were not 
necessarily Indian-owned (or at least were not participating because they were Indian- 
owned) nor did they represent a particular field or industry. Rather, they were 
employers with varied hiring needs that either sought assistance from the grantees or 
were recruited by them. For example, one grantee worked with a small publishing 
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house. This employer found working with the grantee to develop OJT programs made 
good business sense because it kept hiring and training costs low and because the 
employer valued diversity in staffing the firm. 

Although, collectively, grantees offered a wide variety of types of OJT programs, 
most individual grantees reported that they relied on a small number of employers, or 
similar types of employers, with whom they had worked in past program years. In 
general, grantees expressed frustration at the difficulty of attracting new employers. 

This suggests that (1) marketing services to employers and (2) facilitating collaboration 
between employment and training agencies and economic development agencies as 
potential areas for technical assistance. 

Outreach and Recruitment of Employers 

The OJT program demands that grantees conduct outreach not only to 
participants, but to employers as well. Grantees using both service approaches— 
arranging OJTs with diverse employers and targeting specific industries or firms— faced 
numerous challenges in conducting effective outreach to employers who could provide 
OJT opportunities to participants. 

Nonetheless, the challenges tended to be somewhat different, as did successful 
strategies for overcoming them. 

Strategies for Those Providing OJTs with Diverse Employers 

In general, grantees attempting to provide OJTs in diverse slots experienced 
somewhat more difficulty in recruiting employers. These grantees, in an effort to meet 
the specific needs of their participants, attempted to identify OJT opportunities with a 
wide range of employers. They generally used their Indian networks as a base from 
which to recruit employers and planned to expand their range of participating 
employers using these networks. Some grantees developed long-term relationships with 
a cadre of employers that regularly hired or trained program participants. However, 
many grantees were not successful in expanding their networks beyond the Indian- 
owned firms. In several cases, these grantees were located in high-growth, low 
unemployment areas, but were not successful in taking advantage of the good labor 
market to place participants in high-quality jobs. 

Staff reported that there were several reasons their participants were unable to 
reap the benefits of this economic growth including: 
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• Participants lacked skills in demand among local employers. Grantee 
staff reported that, although there were many employers with openings 
claiming that they “will train/no experience necessary,” employers still 
required some work history or demonstration of past performance that 
participants were unable to provide. 

• Participants prefer to work with Indian-owned employers. Grantee staff 
indicated that this was particularly important in urban environments, 
because Indian people who relocated from reservations placed a high 
value on working in an environment in which their cultural norms and 
practices were valued. However, there were few Indian-owned firms in 
most areas, and few opportunities within those firms were available. 

• Participants ' residences were far from areas of high job growth. 

Grantee staff indicated that even in urban areas with overall high 
growth, pockets of poverty exist where the unemployment rates are 
much higher. These are precisely the areas in which participants are 
likely to reside. Because they are often forced to leave their urban 
communities for work, transportation issues emerge as a problem. 

• Employers were reluctant to work with grantees. Grantee staff also 
reported that racism among employers persists and has prevented them 
from developing employer relationships outside the Indian community. 

These challenges are formidable. However, some sites developed strategies to 
overcome them, suggesting that they can be overcome among other grantees in similar 
contexts. 

Bridging Employer-Participant Skills Mismatch. When participants lack the 
skills in demand among local employers, the JTPA program can serve as a bridge 
enabling participants to obtain those skills so that they are qualified for available jobs. 
Programs have demonstrated their ability to bridge this gap by developing their own 
training or by working with a contracted training provider, combining CRT activities 
with work-based learning. For example, one grantee developed an OJT program for an 
individual interested in accounting. The program played an important role in 
connecting the participant and employer to seminars and other training opportunities 
available in or near the worksite. 

In addition, employers with whom we met as a part of the evaluation indicated 
that work-readiness was their foremost concern. This suggests that if JTPA programs 
can develop effective ways to assess participants on the skills employers want, prepare 
participants for work, and then provide technical assistance to employers interested in 
providing additional training, employers may be more receptive to providing 
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participants with training in specific skills or on an ongoing basis. OJT programs can 
serve as an important means toward that end. 

Working with Indian-owned Firms. We heard a great deal about participants’ 
preference for Indian-owned firms. This is a tremendously complex issue. There is 
certainly value in placing participants in Indian-owned firms, particularly participants 
for whom it is important to work in an environment influenced by familiar cultural 
norms. Yet, there was evidence among some grantees that participants’ choices were 
severely limited by this practice. Some grantees directed participants to employment 
opportunities with Indian-owned firms offering only poverty wages, rather than 
searching more broadly for opportunities with a better chance of assisting Indian 
families in achieving self-sufficiency. Alternatively, other grantees directed 
participants to employment situations that forced them to travel long distances or work 
odd hours in order to work in an Indian-owned firm. 

For example, we interviewed one participant who had been placed in an Indian- 
owned firm far from her home. She was forced to travel one hour on two buses each 
way for a wage of $5 per hour to work in the cleaning business, when just a short bus 
ride from her home (and very near the grantee’s offices) was a shopping mall 
advertising positions for $7 per hour, and a gourmet market (with a diverse staff) 
offering $10 per hour. Since these employers were not Indian-owned, however, the 
grantee did not market services to them. This participant finished her OJT program, 
but when her employer, a dry-cleaner, offered her a full-time job, she refused, citing 
her commute and the cost of childcare as major barriers. 

While this was an extreme case, it was not isolated. We heard similar stories 
from other participants, and saw evidence of it in case files. We interviewed 
participants who did express an interest in working for Indian-owned firms; however, 
for most of these participants, this was one of many interests. It is understandable that 
grantees would want to provide their services to Indian-owned firms, but they face 
formidable challenges in developing effective matching and screening procedures to 
ensure that the programs meet the needs of both employers and participants. 

Moreover, the grantees always want to take into account the comprehensive needs of 
participants, considering all potential consequences or unintended outcomes. 

Moreover, many non-Indian employers are actively seeking to diversify their 
workforces to better represent their communities and serve their customers. The INA 
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grantees could play a vital role in bridging this gap by building the capacity of 
employers to better foster diversity in the workplace and by helping participants 
understand the norms of working in a variety of different environments. This in no 
way precludes or even undermines the practice of serving Indian-owned employers, but 
enables the grantees to play a unique and important role in improving the lives and 
livelihoods of participants, and educating majority communities about Indian values and 
traditions. 

Mismatch between Participants’ Localities and Location of Available Jobs. 

The mismatch between the location of jobs and the location of available labor is a 
plight faced by many high-growth urban areas. Recent scholarship on the subject has 
generated considerable attention . 14 This suggests that other organizations providing 
services to individuals seeking work face similar transportation and other challenges 
arising from this “spatial mismatch.” Improved partnerships with other such providers 
may be one way to generate solutions to these shared challenges. For example, one 
grantee began to convene community meetings between the representatives from tribal 
programs and representatives from the three surrounding small towns at the tribe’s 
Family Center. Although not the intent of selecting the venue, attendees, including 
representatives from the local school boards and community colleges, became more 
familiar with the routes on the reservation and could help plan more efficient use of 
shared vehicles (e.g. school buses) or commuting for more than one purpose. 

Employers Reluctant to Work with Grantees. Numerous grantees pointed to 
racism as a major barrier to recruiting new employers. Undoubtedly, this phenomenon 
frustrated numerous grantees, despite their efforts to the contrary. However, there 
were additional reasons cited by employers, and in some cases, grantee staff, for the 
difficulties that emerged when grantees attempted to market OJT services broadly (e.g., 
outside the Indian community). First, employers who had worked with tribal grantees 
(and several grantees themselves) reported that, in some cases, bureaucracy was 
responsible for employer reluctance. In some grantee agencies, each OJT contract had 
to be reviewed by a governing or advisory body associated with the tribe. This was 
time consuming, and in one case, resulted in a lost opportunity to place multiple 
participants in good jobs because the employer, who needed employees immediately, 
was forced to hire independently of the grantee to support business operations. Ideally, 



14 See, for example, William Julius Wilson, The Truly Disadvantaged. 
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reviews of administrative procedures can be conducted among the grantees for whom 
this is a problem, to streamline the process and avoid future misunderstandings. 

Second, employers reported that the paperwork associated with the OJT program 
was burdensome. However, staff experienced in writing OJT contracts can help 
facilitate the paper-trail and ease the burden placed on employers, while ensuring that 
participants needs are not sacrificed in the process. Third, some employers and some 
participants expressed dissatisfaction with their placements. Again, well-trained JTPA 
staff who can conduct high-quality assessments that are not burdensome to participants 
can play an important role in ensuring a quality match and generating mutually 
agreeable expectations on the part of employers and participants. 

Finally, as discussed previously, grantees can play a role in reducing racial 
barriers by demonstrating high-quality program outcomes for participants and 
employers, bridging miscommunication between employers and participants, 
conducting outreach into the non-Indian employer community about the services 
available through the grantee, and establishing partnerships with other service 
providers. These strategies can help grantees serve their communities better by 
enabling participants to access additional services provided by partner agencies and 
providers, and by improving the range and quality of employment and training 
opportunities available through OJT (and other JTPA) programs. 

Strategies for Those Arranging OJTs in Targeted Industries 

In general, grantees using OJTs to support the development of targeted industries 
had less trouble recruiting employers because: (1) employers were often tribally-owned 
and used the JTPA program as its first source of hire, and (2) employers that had 
success with the JTPA program marketed it to other employers— because industries 
were targeted rather than individual employers, word traveled quickly. 

Nonetheless, these sites, too, offer lessons in developing ways to reach new 
employers and develop new OJT opportunities for their participants. Their strategies 
included: (1) working closely with the local or tribal economic development entities, 

(2) focusing on the development of tribal enterprises or tribal agencies providing 
essential services (medical, elder care, etc.), (3) identifying home-based business 
owners that could serve as mentors and provide short-term work-based learning 
opportunities for participants, and (4) developing work opportunities in nearby urban 
areas and ways to support groups of participants in taking advantage of them. 
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• Working with economic development entities. By working closely with 
the economic development entities (local or tribal) , grantees could plan 
to meet the needs of businesses being recruited into their service areas 
by ensuring that work-ready participants were available. Moreover, 
grantees could help facilitate local efforts to develop targeted industries 
in their areas that offer participants current and future employment 
opportunities. 

• Focusing on tribal enterprises. By focusing on the development of 
tribal enterprise, grantees can help support their tribes’ ability to 
prosper within their local areas and reservations. OJT is a particularly 
effective strategy toward this end because grantees can concentrate on 
institutionalizing effective OJT programs with a few employers likely to 
offer jobs and opportunities for advancement on a continuing basis. For 
example, one grantee provided services to a successful timber 
enterprise, and was able to offer participants opportunities at every step 
on their career-ladders because there was consistent promotion, 
advancement, and overall business growth. Other tribes pursued this 
same strategy with their gaming industries. 

• Working with home-based business owners. Although we did not find 
examples of JTPA programs working with home-based businesses, plans 
for such coordination were emerging at several sites. One site, in 
particular, was aware of several local artists who made textiles to sell 
during Indian festivals and at craft shows. Although many participants 
knew basic skills associated with this trade, few had experience 
handling the business side. Since it is a seasonal business, JTPA staff 
were attempting to develop an OJT program whereby a participant 
would assist these artists in their trade and in the management of their 
businesses (finance, marketing, distribution, etc.) during their busy 
seasons, in hopes that these OJT participants would be interested in 
starting their own businesses. 

• Developing off-reservation opportunities. In areas with minimal 
opportunities for employment on-site (on the reservation or locally) , 
some grantees were exploring ways to develop OJT programs for 
groups of participants who could then travel together to their places of 
employment and return home together as well. This strategy would 
enable the participants to earn higher wages than they would locally, 
and would allow them to remain living “at home” (whether on 
reservations or in urban communities) and commute with a community 
of peers. 

The diversity in approaches and outcomes makes it impossible to provide an 
overall assessment of recruitment and outreach vis a vis the high-quality model of 
services. However, the relative lack of knowledge about strategies used by other sites, 
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combined with the overall enthusiasm for peer-to-peer sharing as a means of technical 
assistance demonstrated by grantees (as discussed in Chapter III) , suggest that 
additional opportunities for communication between grantees and grantee staff would 
benefit IN A programs, and OJT programs in particular. 

Quality Designs for Learning Objectives and Monitoring 

The OJT program is one the most difficult to implement well, because it requires 
significant and ongoing coordination between the participant, the employer, and grantee 
staff. The program is particularly difficult to implement in areas served by INA 
grantees for many of the reasons already cited throughout this report, including: (1) the 
absence of private-sectors employers, (2) the lack of infrastructure that could enable 
participants to reach jobs, (3) the inadequacy of funds available to serve all eligible 
customers, and (4) the importance of cultural context and values in the workplace and 
community. However, many grantees have developed strategies that enable them to 
provide OJT services consistent with the model of high-quality services described in 
Chapter I, despite these challenges. Some of these strategies have been discussed 
throughout this chapter, including methods for recruiting employers, developing sound 
training plans, and so on. In general, however, the development of clear training 
objectives and the provision of effective supervision proved the most difficult for 
grantees to implement well. 

Developing Clear Training Objectives 

Much of the difficulty grantees experienced in developing clear training 
objectives for OJT programs arose from their local contexts. For grantees that 
provided OJTs under very difficult employment conditions, determining clear training 
objectives proved a challenge because grantees were never certain that employers would 
offer work to participants at the completion of their training programs, despite the fact 
that this was written into the OJT contract. 15 Grantee staff writing OJT contracts under 
these circumstances wanted to ensure that the participant learned enough to become 
valuable to the employer, who would then be more likely to hire the participant, but 
also learned more general transferable skills so that the participant would be better able 
to secure work elsewhere. This was a difficult challenge and sometimes resulted in 



15 In some cases, this was clear abuse of the OJT program on the part of employers. However, 
in other cases, a downturn in the local economy made it impossible to hire participants even after they 
successfully completed training. 
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training plans that were unclear or difficult to follow from either the employer’s or 
participant’s perspective. Grantee staff must work closely with employers and 
participants to strike a healthy balance and coordinate with other tribal entities to the 
extent possible, to enhance the ability of employers to follow through with hiring 
commitments. 

Service approaches used by the grantees also created challenges in establishing 
training objectives. For example, grantees providing OJTs with diverse employers 
faced difficulty in establishing clear training objectives because of the time constraints 
of program staff. Because staff were working with multiple and diverse employers, the 
time grantee staff could spend with individual employers was reduced. In some cases, 
this meant that once the OJT contract was signed, grantee staff were not always able to 
follow-through and develop high-quality training plans. As a result, employers had 
greater control over the training in which participants engaged. Employers that had 
developed effective training practices ensured that participants received high-quality 
training. However, other employers did not establish formal training programs at all. 
Grantees face a challenge in allocating adequate productive time to spend with 
employers and participants involved in OJT programs to ensure that those programs run 
efficiently and produce intended outcomes. 

Similarly, grantees supporting targeted industry development faced parallel 
challenges in identifying clear training objectives. Because OJT programs developed to 
support industries tended to evolve into a situation in which employers made a specific 
number of “slots” available for participants for a given period of time, the training 
plans themselves tended not to be particularly individualized. Although the 
arrangements did tend to be institutionalized— employers were familiar with the 
requirements of providing high-quality training and developed processes through which 
this could be accomplished— they did not necessarily require that individual training 
plans be developed. As a result, when participants may have desired specific outcomes 
from their training programs, they were not necessarily accommodated. 

Supervising OJT Participants 

Regardless of their approach, grantees were challenged in their efforts to develop 
effective methods with which to supervise participants, or coordinate with participants’ 
on-site supervisors, to ensure that participants’ particular needs and aspirations were 
taken into account in implementing their OJT programs. The high-quality model 
suggests that effective supervision may take the role of coaching or mentoring. Several 
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of the participants with whom we met indicated that such mentoring had taken place 
during their programs. However, we found little evidence to suggest that grantee staff 
played a role in facilitating this process. Although grantee staff may take a minimal 
role in following up to ensure that supervision of participants is effective among 
employers for whom this practice in common, they could certainly play an important 
role in building the capacity of other employers to promote these practices. 

System Level Influences on Quality of OJT Programs 

Funding constraints and the community context loomed as two important system- 
level factors that influenced the use and quality of OJT. 

Funding Levels 

Perhaps the most significant impact that funding— specifically, decreased funding 
over time— has exerted on grantees is that it has forced them to make difficult choices 
about the use of resources to support their programs. OJT in particular has been 
negatively affected because (1) it is perceived as an expensive program among most 
grantees because of the wage reimbursement, and (2) it can be paper- intensive, and 
therefore labor-intensive, which makes the cost of administering the program 
potentially burdensome as well. As noted previously in this chapter, inadequate 
funding was the number one issue cited as a barrier to providing OJT services among 
grantees that did not offer OJT, and two of these sites had recently eliminated the 
service as a specific response to budget reductions. Among the grantees that do offer 
OJT services, funding was also cited as a constant challenge in providing these 
services. 

However, many of these same grantees acknowledged that they emphasized WEX 
or CSE programs, which can be as or more expensive, and even though OJT programs 
promise full-time employment after the initial training period, and therefore are more 
likely to facilitate participants’ ability to achieve self-sufficiency. Of course, in some 
areas, the absence of stable employment opportunities may make effective OJTs 
difficult, but, even under these circumstances, OJTs may be used to support essential 
business development. Because grantees are interested in documenting the individual 
and community outcomes of their programs, they may benefit from considering the use 
of OJT in this way. 



VII-39 



213 



Community Context 

The size, location, and characteristics of the service population and local 
economy have significantly impacted grantees’ ability to implement high-quality 
programs and achieve intended outcomes. Many grantees expressed frustration at the 
frequency with which employers’ needs changed or other problems emerged that 
prevented employers from following through on their commitments to hire OJT 
trainees. Similarly, the absence of active labor markets in many areas, or at least of 
employers in proximity to participants, raises viability and sustainability issues. At the 
same time, OJT designed to support target industries reduce individual choice. 

Another difficult contextual challenge emerged among tribal grantees providing 
services under TERO (Tribal Employment Rights Office) guidelines. The TERO 
offices were often located in or near the tribal Employment and Training Offices, and 
staff generally worked closely together. The TEROs’ mandate was to ensure that 
services were provided with appropriate attention to the employment rights of tribal 
members; the offices thus functioned much like affirmative action offices common in 
many other places. Because OJT services consumed significant portions of grantees’ 
funding, the TERO staff often wanted to ensure that tribal members were served first. 
While grantee staff were unanimously supportive of TERO policies, these policies 
complicated the task of matching participants to jobs because grantee staff had to ensure 
that they placed all qualified tribal members eligible for services before they placed 
other eligible participants. Sometimes this caused tension in service areas characterized 
by high unemployment and few opportunities, particularly when the services in 
question were CSE, WEX, or OJT — all resource-intensive service activities. 

Despite these factors, however, many grantees were able to develop creative 
approaches to developing and implementing OJT programs with a wide range of 
employers, or in support of the development of specific industries, and diverse groups 
of participants. Increased opportunities to share these practices may better enable 
grantees to learn from each other’s successes and promote effective practices more 
widely. 

Outcomes 

Overall, the quality of OJT services and their outcomes was mixed. Grantees 
faced numerous challenges in implementing these programs. The most important of 
these, regardless of the grantee’s approach or local context, was balancing the needs of 
participants with those of employers, and those of the grantee (in terms of resources 
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and staff). When these elements were in balance, grantees were more likely to develop 
quality OJT programs that satisfied multiple key stakeholders. For example: 

One grantee wrote an OJT contract to train a participant to be an assistant 
manager of a small tribally-owned retail business. This particular program was 
an excellent match for the participant. Although she had limited work 
experience, she was highly motivated and enjoyed the opportunity to work with 
the public. A close relationship between her, her case manager, and her 
supervisor resulted in a detailed training plan, and instruction on how to ask for 
promotions, pay increases, and professional development opportunities. At the 
time of our visit, this participant had recently taken part in an out-of-state 
professional development program for managers. She had learned a great deal 
and had clearly raised the level of professionalism within the entire establishment. 

Similarly, we met with a participant who was working as a manager in one 
of the few retail businesses on a tribal reservation. She had entered her first job 
as an OJT participant, and was now returning to the program as recruiting 
additional staff for her employer. Since she had been a participant herself, she 
recognized the importance of working with the grantee and program participants 
to develop training plans. She did this in an effort to provide other program 
participants with opportunities similar to those she was offered. When we 
interviewed the OJT participants who worked for her, they indicated that they had 
taken active roles in their own training programs, and that their manager 
regularly encouraged them to take on increasing responsibility and ask for 
promotion and training opportunities. Although some of these participants had 
enrolled in their training programs without the intention of turning them into 
long-term professional careers, it was clear that many of them would indeed 
continue on career paths. 

These examples point to the capacity of the OJT service activity to provide 
opportunities for training and potential long-term employment to participants, while 
simultaneously supporting the more general goals of strengthening Indian communities. 

Conclusion 

This chapter provided an in-depth look at the practice, challenge, and promise of 
OJT programs among the 23 grantees we visited. These sites face clear challenges in 
delivering high-quality OJT programs in their communities, including: 

• Challenges that arise as a result of the service environment, such as a 
lack of private sector employers, underdeveloped transportation 
infrastructure, and a low- wage labor market. 
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• Challenges that arise from circumstances internal to the grantee, such 
as staff capacity issues, participants’ barriers to employment, or 
excessive tribal bureaucracies. 

• Challenges arising from systemic factors, such as inadequate funding, 
inadequate MIS support, or confusing rules and procedures demanded 
by the JTPA program. 

However, the ability of some of the grantees with whom we visited to overcome 
these challenges and develop OJT programs that meet participants’ needs is 
commendable. Additional technical assistance and peer-to-peer sharing on strategies to 
meet employer needs, promote community development, develop improved training 
plans, and conduct efficient and effective follow-up may help these grantees realize the 
full promise of the OJT program, and better serve a wider range of participants and 
employers in their communities. In so doing, grantees and participants can reap the 
full benefit of OJT, which can constitute a highly effective strategy for forging strong 
relationships with the employer community. 
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VIII. WORK EXPERIENCE AND COMMUNITY 
SERVICE EMPLOYMENT 



This chapter describes two work-based activities— Work Experience (WEX) and 
Community Service Employment (CSE)— under JTPA’s Title IV Section 401 IN A 
program. WEX and CSE programs provide opportunities for participants to acquire 
occupational skills and hands-on work experience while offering them the advantage of 
gaining immediate income. These activities also benefit employers by providing them 
with subsidized labor in exchange for training participants on the job. WEX and CSE 
activities in INA programs are thus unique because they enable participants to enter the 
labor force and allow employers to utilize subsidized employment opportunities to 
support their human resource needs. These programs can also promote local economic 
growth and vitality by subsidizing staffing and training costs in tribal agencies and other 
public sector firms in local communities. 

In this chapter, we discuss findings resulting from on-site visits to the 23 case 
study sites and in-depth observations of 13 WEX programs and 4 CSE programs. Our 
analysis is based on: the observations of worksite activities; discussions with INA 
program staff and worksite supervisors; and case file reviews of and discussions with 
current and former WEX and CSE participants. This chapter first provides an 
overview of WEX and CSE activities. Second, we discuss the degree of emphasis of 
WEX and CSE in the 23 case study sites. Third, we address the objectives of WEX 
and CSE among grantees providing these services. Fourth, we describe the design and 
delivery of WEX and CSE programs and also describe the factors influencing the 
program designs. We also discuss how grantees selected worksites, the types of 
participants enrolled in WEX and CSE, the training options available, and the intensity 
and duration of the service. Finally, we assess the effectiveness of these programs 
using the model of high-quality training. 

Overview of WEX and CSE 

The purposes of WEX programs include helping participants gain work 
experience and learn basic work-related skills. Work Experience assignments are 
typically arranged by the grantee and are generally only allowable in the public sector, 
including non-profit community based organizations and tribal and other governmental 
agencies, although in some circumstances work assignments in tribal enterprises are 
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also permitted. Grantees typically target participants with limited work experience and 
place them in entry-level work assignments for a maximum of 1 ,000 hours in one 
program year. Participants can enroll in WEX for part-time or full-time employment, 
but the length of WEX participation varied widely among case study grantee — from 
several days to 26 weeks— depending on the needs of the participants and the 
participating employers. Participants are typically paid minimum wage for their 
training and all wages are paid by the grantee. Although employers are not expected to 
hire WEX participants at the termination of the service , grantees strongly encourage 
them to do so. 

Several features about WEX activities in INA programs are unique. First, many 
case study grantees view WEX activities as the primary vehicle to develop participants' 
job readiness skills while earning needed income at the same time. Second, because of 
limited employment opportunities in rural areas, especially reservations, case study 
grantees often use WEX services to temporarily staff tribal agencies to relieve the strain 
on tribal resources. These grantees view WEX activities as beneficial to both 
participants and their communities because savings from staffing costs can support 
other community activities and services. Third, WEX and CSE programs are often the 
only means by which participants can gain work opportunities and acquire knowledge 
about demanding occupations in local areas. 

Community Service Employment (CSE) is unique to Title IV Section 401 
programs but, like WEX, also provides opportunities for participants to gain work- 
related skills in subsidized, entry-level jobs. CSE work assignments typically include 
work activities within the public sector but can include work related to tribally- 
sponsored income generating activities so long as the profits go back to tribal revenue. 
Similar to WEX programs, participants can work part-time or full-time and are paid at 
least the minimum wage by the grantee. Employers are not expected to hire 
participants at the end of the work assignment but, again, grantees highly encourage 
this. 



All case study grantees offer WEX and/or CSE as service options to participants. 
Many case study grantees, particularly tribal grantees, that offer both WEX and CSE 
programs design these activities in a similar fashion, and even use the same worksites 
for both WEX and CSE. Other grantees distinguish these activities by the kinds of 
work assignments developed for participants. For example, many case study grantees 
use WEX assignments for entry-level employment opportunities, but reserve CSE work 
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assignments for more experienced and job ready participants and, thus, consider CSE 
as a “step up” from WEX. Overall however, there is significant overlap in the design 
and delivery of WEX and CSE in INA programs , so we will discuss these two 
programs jointly, noting differences where relevant. 

Extent of Work Experience and Community Service 
Employment 

All case study sites provide WEX and CSE services, but the extent to which 
grantees emphasized the use of WEX and CSE varied considerably. At one extreme, 
grantees use these programs sparingly because these services are expensive and would 
absorb a large portion of the JTPA budget. At the other extreme, some grantees used 
WEX as the primary service option. Table VIII- 1 describes the level of emphasis on 
WEX and CSE activities by the percentage of participants enrolled in WEX and CSE 
and the number of grantees offering these activities. 1 

Table VIII-1 

Percentage of WEX and CSE Enrollment 
by Number of Sampled Grantees PY 95 



Percentage of 
Participants 


# of Grantees 
Offering WEX 


# of Grantees 
Offering CSE 


0-10% 


5 


17 


11-20% 


6 


6 


21-40 


5 


0 


Above 40% 


7 


0 


Total 


23 


23 



As the table shows, the emphasis on WEX services differed significantly across 
the 23 grantees. Seven grantees from our sample established WEX as a dominant 
service activity in which 40% or more of its participants were enrolled. 2 Five of these 



1 Percentages of participants enrolled in WEX and CSE services are drawn from the Annual 
Status Report in PY 95. 

2 One tribal grantee located on a reservation enrolled 80% of its participants in WEX in PY 95. 
The heavy reliance on WEX stems from limited alternative training opportunities and private sector 
employers to support on-the-job training. 
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are tribal grantees. At the other extreme, five out of the 23 grantees only infrequently 
provide WEX services: 10% or fewer of their terminees were enrolled in this service 
option in PY 95. One out of the five is a tribal entity, and the rest are non-tribal. 
Finally, eleven case study grantees established a moderate emphasis on WEX: 11% to 
40% of their terminees received WEX. These grantees are split between half tribal and 
half non-tribal entities. 

Based on the 23 case study sites, we found that tribal grantees tend to emphasize 
WEX activities more so than non-tribal grantees. For example, 5 of the 7 grantees (or 
71%) offering WEX to more than 40% of their participants are tribal. At the other 
extreme, only 1 of the 5 (or 20%) offering WEX to 10% or fewer of their participants 
are tribal. 

The degree of emphasis for CSE differed significantly from WEX. In general, 
CSE activities do not make up a significant portion of the service design within our case 
sample. As Table VIII- 1 describes, 17 out of 23 (74%) grantees (both tribal and non- 
tribal) offer CSE to fewer than 10% of their participants. Six of these grantees do not 
offer CSE at all, because they see these services as either too expensive or overlapping 
significantly with WEX activities and they thus preferred not to provide CSE as a 
separate service activity. At the other extreme, one grantee, a tribal grantee, provided 
CSE to 19% of its terminees for PY 95, which was by far the highest proportion of 
CSE used in our case study sample. None of the sampled grantees provide CSE as 
their primary service option. 

Grantees not emphasizing WEX and CSE programs cited the following reasons 
for their limited use: 

• Limited funding to support work-based activities. Because grantees pay 
100% of participants’ wages for WEX and CSE activities, grantees have 
restricted the use of work-based activities to reduce the strain on the 
JTPA budget. 

• Limited subsidized work opportunities in local areas. Given restrictions 
on the use of WEX and CSE, grantees have faced difficulty identifying 
public sector employers who have slots available. JTPA staff have 
faced significant challenges developing WEX and CSE positions 
appropriate for their participants 

• Overlap between CSE and WEX. Grantees that limit the use of CSE 
offer WEX services and feel that only one work-based activity is 
sufficient to address participants’ work needs. 
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• WEX and CSE services yield modest outcomes. Although WEX and 
CSE positions provide immediate income for participants and have the 
potential to increase participants’ occupational skills, the lack of assured 
job placements at termination deterred some grantees, particularly non- 
tribal grantees, from focusing on their use. 

By contrast, the emphasis that some grantees placed on WEX as the main service 
activity stems from several factors, including: 

• The absence of economic development and employment opportunities in 
local areas. Most grantees emphasizing WEX services as the dominant 
mode of service delivery are operated by tribal governments located on 
and off reservations. The weakness of the local economy and the lack 
of private sector employment in many Indian communities constrained 
the grantees’ ability to provide alternative service options. 

• Participants ’ expressed need for immediate income gained from 
working. Most participants seeking JTPA services in Section 401 
programs prefer immediate employment to occupational and basic 
education training. To address the economic needs of participants, 
grantees provided subsidized employment services accordingly. 

• Limited service providers for classroom training. Grantees operating in 
remote areas faced challenges in providing classroom training as a 
service option because of the lack of classroom training providers in the 
local areas. 

• Participants ’ lack of adequate job skills necessary to gain unsubsidized 
employment. Most participants lack adequate job skills and a solid work 
history to find permanent employment on their own. WEX work 
assignments enable participants to establish a work history and enter the 
labor force. 

The emphasis on subsidized employment programs in some INA programs reveals 
several challenges endemic to Section 401 grantees, including local economies 
dependent on government subsidies, unstable market environments in local areas, and 
the absence of private sector employment opportunities to promote productive 
enterprises. These conditions foster a continued reliance on the services of WEX and 
CSE to sustain the economic well being of participants and their communities. 

Goals and Objectives for WEX and CSE 

The primary goal of WEX and CSE activities is to provide participants with 
employment opportunities by assigning them to work assignments that closely match 
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their skill levels and career interests. In addition to this goal, grantees identified a 
broad-range of other goals and objectives for WEX and CSE programs. These include: 

• Providing an exposure to the world of work. Many grantees, 
particularly those located on reservations, view WEX and CSE as 
important to providing participants an exposure to the world of work. 

Grantees reported that WEX and CSE services are intended to instill a 
solid work ethic and provide hands-on experience so that participants 
can better understand the dynamics of the world of work. Grantees that 
emphasized "learning by doing" saw WEX and CSE experiences as 
essential first steps towards obtaining permanent employment for their 
participants. 

• Acquiring occupational skills. Gaining occupational skills is an 
important goal identified by grantees offering WEX and CSE programs. 
Although this goal is not always realized, grantees perceive WEX and 
CSE programs as a means to upgrade participants’ existing work skills 
and develop new skills for workers with a limited work history. 

• Providing income to participants who are seeking permanent 
employment or are enrolled in another service activity. Providing 
income to participants through WEX and CSE services was the most 
common goal across study sites, but was especially important to 
grantees operating in areas with virtually no employment opportunities. 

• Facilitating placement in long-term unsubsidized employment. Although 
employers participating in WEX and CSE are not required to hire 
participants at the completion of services, grantees often encouraged 
them to do so. 

• Building participants' self-esteem. Increasing self esteem with 
subsidized employment opportunities remains an important goal for a 
number of grantees, especially tribal grantees located on reservations, 
where participants lack occupational skills and the self-esteem to find 
work on their own. Grantees emphasized the need to give participants a 
“taste of success” so that they can better understand their potential in 
the labor market. 

• Supporting traditional Indian and Native American values. Some 
grantees, especially non-tribal grantees, perceive WEX and CSE 
activities as opportunities to inculcate traditional Native American 
values. Three grantees in particular encouraged WEX and CSE 
participants to learn the history of Indian culture as part of their 
WEX/CSE assignments; and, in at least two sites, participants are 
required to speak and understand tribal dialect to be eligible for certain 
WEX/CSE work assignments. Integrating culturally specific activities 
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into WEX and CSE services was one way to support INA values in the 
workplace. 

Grantees also perceive WEX and CSE programs as benefiting their communities 
in addition to individual participants. Three community-level goals noted most 
frequently included: 

• Staffing the JTPA program, tribal enterprises, non-profit organizations, 
and public agencies. WEX and CSE activities are often used to defray 
the costs of staffing a variety of organizations, including various 
departments operated by the grantee, tribal government, non-profit 
organizations and a host of public service agencies such as the Bureau of 
Indian Affairs, Indian Health Services, and state agencies. Grantees 
implicitly established this goal to benefit participants and worksites. 

• Promoting community and economic development. An important overall 
goal identified by grantees is to facilitate community and economic 
development by coordinating WEX and CSE services with the needs of 
the local community. To achieve this goal, most sites utilized the tribe 
or other Indian-friendly organizations as worksites. 

• Establishing relationships with employers. Because WEX and CSE 
programs benefit employers and participants, these programs can 
facilitate the hiring process and encourage employers to hire WEX and 
CSE participants directly from the program. 

Although these general goals were most frequently reported by grantees offering 
WEX and CSE, specific objectives varied enormously depending on participants’ and 
employers’ needs. 

Designing and Delivering WEX and CSE 

In designing WEX and CSE activities, case study grantees were influenced by 
several different factors, including the employment outlook of the local area, the needs 
of the participants, and the grantees’ ability to develop collaborative relationships with 
employers. Each of these factors influenced the design of WEX and CSE across case 
study sites and enabled grantees to design these activities in a variety of ways using 
diverse strategies. 

To ensure that participants’ diverse experiences and needs were addressed 
appropriately, grantees designed WEX and CSE activities in several ways, including: 
(1) providing WEX and CSE services as “stand alone” activities, (2) combining WEX 
and CSE services with each other, and (3) integrating WEX and CSE services with 
another service activity or activities. All sampled grantees use all three approaches at 
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different stages of the service planning, depending on the occupational needs of the 
participants and the employment outlook of the local area. We discuss the different 
approaches below. 

Providing WEX and CSE as “Stand-Alone” Activities 

The first approach case study grantees used to deliver WEX and CSE services 
was to offer them as stand-alone activities that function independently of other service 
options. Although all sampled grantees offered WEX and CSE services alone, those 
emphasizing this approach strongly believed that WEX and CSE are the kinds of 
services for which their participants had the greatest need because of limited training 
and job opportunities in their local areas. 

Although the focus on either WEX or CSE alone did not offer participants the 
opportunity to explore other types of training options, and, in many cases, had modest 
occupational training objectives, these services provide participants an income and can 
increase their work experience and knowledge of the world of work. Most participants 
enrolling in WEX or CSE as stand-alone activities typically possess low skill levels and 
are generally long-term unemployed, or had been underemployed and are seeking to 
increase their job opportunities in different industries. 

Case study sites providing WEX and CSE services independent of other service 
options generally enrolled participants for the maximum allowable hours for WEX and 
CSE, unless participants found employment prior to the end of the time limit for these 
services. 

Combining WEX and CSE 

The second approach to providing WEX and CSE is to pair them together in one 
service package. This approach is intended to accomplish several objectives, including 
(1) enhancing participants’ employability, (2) extending the duration of paid work- 
experience to satisfy participants’ economic needs, and (3) attracting employers to hire 
participants at the completion of on-site customized training. Grantees in this category 
typically offer these services sequentially, starting the service package with WEX 
activities for 1,000 hours, then proceeding to CSE activities. Work assignments 
generally remain unchanged regardless of participants’ shift in status from WEX to 
CSE, although sometimes changes are made to accommodate participants’ experiences 
at the worksite or employers who no longer need their services. 
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Combining WEX and CSE activities sequentially can offer clients the opportunity 
to enhance their skill sets with one employer for a lengthy period of time. However, 
data from case files and interviews with participants and program staff reveal that 
enrolling in lengthy work assignments through a combination of WEX and CSE 
activities was rarely linked to acquiring new or additional work skills. Participants 
were typically not offered additional responsibilities with their status change. The 
exception to this occurred in worksites where supervisors established procedures to 
enable participants to progress to challenging tasks with the status change from WEX to 
CSE. For example, one worksite allowed CSE participants to take responsibility for 
more complex tasks at the tribal court and to “move up” to a law clerk position from a 
clerical position upon completion of WEX hours. 

A number of grantees provided a combination of WEX and CSE services to 
employers as an incentive to hire participants on a permanent basis. Indeed, some 
worksite supervisors used WEX and CSE experiences as a means to measure 
participants’ work habits before deciding whether to hire participants at the termination 
of the work assignment. However, an extension of WEX and CSE activities did not 
always mean that participants were assured unsubsidized employment, because 
employers often lacked the capacity to hire additional staff. This scenario was evident 
in several case study sites where several participants enrolled in WEX and CSE in the 
hopes that the worksite supervisors would hire them on a permanent basis once they 
completed their WEX and CSE assignment. Nonetheless, participants reported that 
they gained extensive work experience from their work assignments. 

Integrating WEX and CSE with Other Service Activities 

The third approach that sampled grantees used is to combine WEX or CSE with 
other service activities such as classroom training, training assistance, or on-the-job 
training, either sequentially or concurrently. The combined service approach allows 
participants to receive more than one service while earning income in the interim. 
Integrating multiple services varied considerably across sites depending on participants’ 
service needs or the grantee’s capacity to support multiple service options. 

When WEX and CSE activities are provided sequentially with other service 
activities, participants typically enroll in work assignments after they completed training 
assistance or occupational or basic skills training, and while they are awaiting 
permanent job placement. We observed examples of work assignments that were 
directly related to participant’s career interests and goals, including the following: 
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One grantee carefully selects WEX participants to work in its art gallery 
and screens applicants for a variety of characteristics, including interest in 
Native American fine art, strong work ethic, and ability to perform several 
tasks simultaneously. Participants are also expected to show a strong 
interest in the history and culture of the Indian experience and are required 
to read materials to enhance their understanding of Native American 
culture. WEX participants assigned to the art gallery tend to pursue a 
career related to fine art. 

WEX and CSE services are also sometimes provided concurrently with another 
service activity, including classroom-training or training assistance. The advantage to 
this approach is that it sometimes enables participants to combine hands-on, practical 
training with theoretical concepts gained in classroom training. Thus, several sites 
allow participants to enroll in a GED class or an Associate of Arts Degree program in 
the evenings while working in WEX or CSE assignments during the day. For example, 
one grantee enrolled participants in evening child development courses while they were 
working in a child care center as WEX participants during the day. In another case: 

A grantee located on a reservation enrolls participants interested in pursuing 
a career in the medical service profession in its emergency medical 
technician (EMT) training. This WEX and combined classroom training 
program involves extensive classroom training on the principles of medical 
care and includes a practical, hands-on component that requires participants 
to observe real-life examples of the responsibilities of an EMT as carried 
out in an ambulance. Most participants who enrolled in this activity became 
employed as EMTs at the end of the training. 

Another advantage to this approach is that participants can gain vocational or 
basic skills training, even if unrelated to the work assignment, while earning an 
income. The combined approach to WEX and CSE also ensured that participants learn 
specific occupational skills outside of their work assignments. 

The main limitation to this approach is the high cost associated with combining 
multiple service options, which diminishes the funds available to support other service 
activities. Paying all training wages, in addition to classroom training, is expensive for 
grantees, and efforts to develop jobs for WEX and CSE participants also consume 
considerable staff time. As a result, some grantees limited the use of multiple service 
options to only those participants facing unusual circumstances. For example, a grantee 
with moderate funding only provides a combination of WEX and CSE services with 
another service activity to those deemed “hardest to serve.” 
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Challenges to Designing Quality WEX and CSE Activities 

Grantees face several challenges as they design WEX and CSE activities relevant 
to the needs of their participants and their communities. These key challenges include: 

• Sustaining funding sources to operate WEX and CSE services. A 
number of grantees limited the use of WEX and CSE activities because 
of financial constraints, even though participants expressed strong 
preferences for these activities. Most tribal grantees, however, have 
been able to leverage non-JTPA funds to continue to offer WEX and 
CSE opportunities to ensure that participants have opportunities to 
transition to self-sufficiency. 

• Ensuring that participants acquire new work skills to complement their 
existing skills through clearly defined training activities. Many WEX 
and CSE work assignments offer only modest training objectives, even 
to those in long-term work assignments. Because these activities are 
intended to provide valuable work experience and increase participants’ 
potential in the labor force, identifying work assignments with clear 
training goals proved to be a challenge for some grantees. 

• Providing services consistent with participants’ career goals and skill 
levels. A number of grantees assign WEX and CSE participants to 
worksites when positions become open and in some cases, neglected 
participants’ professed occupational interests. A number of participants 
reported that they were placed in WEX and CSE assignments “wherever 
there are jobs." 

• Developing collaborative relationships with a wide range of employers. 

Some grantees faced difficulty in developing relationships with 
employers who could offer WEX and CSE training, especially those 
outside of tribal agencies, and thus often relied on the same few 
employers to support WEX and CSE programs. 

Worksites for WEX and CSE Programs 

The mode of providing WEX and CSE services differed somewhat among 
sampled grantees. While the majority of work assignments are provided in-house by 
the grantee or the tribe, a number of grantees established relationships with public, non- 
profit agencies to function as worksites for WEX and CSE programs. This section 
discusses how grantees delivered WEX and CSE services in the local areas, including 
the challenges in selecting worksites, the types of worksites targeted, and the factors 
influencing the selection of worksites. 
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Challenges 

In selecting worksites for WEX and CSE services, INA grantees reported several 
challenges. These include: 

• The limited availability of worksites. Grantees located in remote areas 
faced tremendous challenges in identifying employers to participate in 
WEX and CSE activities. The limited number of service providers in 
some areas meant that tribal grantees relied on the tribe as the sole 
worksite for WEX and CSE activities, while non-tribal grantees relied 
on the same service providers year after year. To address this 
limitation, grantees established strong working relationships with the 
few WEX and CSE providers located within their communities. 

• The inaccessibility of worksites. A repeated challenge faced by grantees 
located in isolated areas is the participants’ inability to access worksites, 
many of which are located great distances away from the JTPA office. 
Although the majority of tribal grantees selected worksites within 
reservations, other tribal grantees selected some worksites outside of the 
reservation: one grantee assigned WEX and CSE positions in the closest 
city located 65 miles from the reservation. Another tribal grantee with 
multiple field offices placed WEX and CSE participants in the closest 
metropolitan area, while yet another grantee broadened its selection of 
worksites to include employers in the closest major city. 

• The limited ability of worksites to hire participants at the completion of 
the work assignment. While a fair number of grantees indicated that a 
number of employers were willing to offer WEX and CSE services, 
grantees noted that it was difficult to identify worksites that had the 
capacity to hire participants at the completion of services. 

Given these challenges, grantees adopted a number of strategies for selecting 
worksites. We discuss these strategies below. 

Strategies for Selecting Worksites 

The procedures for identifying and selecting worksites varied considerably across 
sites. Selecting appropriate worksites for WEX and CSE services is critical to ensuring 
high quality, responsive services, according to our model of high-quality training 
presented in Chapter I. Grantees relied on a broad-range of worksites to provide WEX 
and CSE services, including tribally controlled organizations, the JTPA grantee itself, 
community-based organizations, community colleges, churches, non-profit 
organizations, and public agencies. To address the common challenges presented 
above, grantees adopted several strategies in selecting worksites, including: 

• Selecting worksites from the local areas, including tribal enterprises. 
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• Providing WEX and CSE services in-house 

• Establish a competitive worksite selection process that requires 
worksites to submit proposals to train WEX and CSE participants. 

Selecting Local Worksites, including Tribal Agencies and Enterprises. All 

tribal grantees in our sample arranged placements in local worksites, primarily with 
tribally controlled agencies and enterprises, including different departments within 
tribal government, such as tribal court, health services, and information systems. 

Several reasons account for why grantees rely heavily on tribal organizations to 
serve as worksites. Grantees reported that selecting tribal agencies benefits participants 
by: (1) providing them opportunities to participate in “comfortable” work experiences 
with other Indians and gain mentorship opportunities with experienced Indian workers, 
(2) enabling participants to get “a foot in the door” to permanent employment with the 
tribe, where positions are generally scarce and competitive, and (3) making work 
assignments convenient for participants by placing participants in worksites close to 
their homes on the reservation or in the local communities. 

In addition to benefiting participants, grantees reported that using tribal agencies 
as worksites provides substantial benefits to the local communities by alleviating 
staffing and training costs of tribal agencies. For example, participants often work at 
Indian Health Services in a wide variety of positions, including as: a clerk in medical 
records or patient intake services, a medical assistant, an emergency medical 
technician, and a certified nurse’s aide. Similarly, a number of WEX participants also 
staff a tribally- run day care center. Grantees reported that staffing these community 
institutions with WEX and CSE participants is not only vital to the sustainability of 
these organizations but also supports overall self-sufficiency for individuals and their 
families by reducing staffing and training costs. 

Although there are clear community benefits to using tribal agencies as worksites, 
decisions about using which tribal agency was sometimes influenced by tribal politics. 
For example, one tribal grantee believes that certain departments within tribal 
government are “due” a number of WEX and CSE participants, because of personal 
relationships between JTPA staff and staff from tribal agencies. Moreover, some WEX 
and CSE designs appeared to be oriented primarily to further the objectives and meet 
the needs of tribal institutions, and only secondarily to meeting the training needs of 
participants. For example, respondents reported that they assigned WEX and CSE 
participants to tribal organizations primarily because they wanted to help the tribe in its 



staffing needs. In fact, several grantee staff members acknowledged that a number of 
WEX and CSE assignments could have taken place as direct hires, without WEX and 
CSE subsidies, but the grantee wanted to “meet the needs of employers. ” Although 
this strategy was not practiced among all tribal case study grantees, there was evidence 
to suggest that tribal politics and the needs of the tribe greatly influenced the kinds of 
worksites selected. 

Finally, in addition to using tribal agencies, a number of tribal grantees also 
sought other local worksites that were unaffiliated with the tribe. This was necessary 
because of the limited positions in tribal agencies. For instance, one tribal grantee 
established a strong working relationship with local city personnel offices so that 
participants interested in working for the city government could access these jobs 
through WEX and CSE. 

Providing WEX and CSE in-House at JTPA Offices. To address the challenge 
of limited worksites in some local areas, a number of grantees provide WEX and CSE 
activities in-house at various capacities ranging from clerical support to custodial 
positions. The quality of these work assignments was decidedly mixed. In some cases, 
grantees assigned participants to positions within the JTPA office that provided valuable 
training in modern office procedures, such as filing, accounting, and answering phones. 
In other cases, the positions offered minimal opportunities for participants to gain or 
practice job skills, but they nonetheless were provided needed temporary income while 
undergoing job search. These positions also enabled the grantee to carry out its 
necessary office procedures at minimal expense. 

Establishing a Competitive Worksite Selection Process. We found that in some 
case study sites grantees were very careful in selecting and recruiting worksites, and 
JTPA staff even assessed employers’ ability to provide adequate training needed by 
participants. Four grantees established a competitive employer selection process and 
evaluated the employers’ track record in hiring participants upon completion of WEX 
and CSE services. These grantees carefully screened worksites and requested them to 
submit a proposal for WEX and CSE work assignments for JTPA staff to review. This 
procedure was designed to ensure that participants were placed in high-quality positions 
and, ideally, attained permanent positions upon completion of the work assignment. 
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Factors Influencing the Selection of Worksites 

A myriad of factors are important in shaping the decisions in selecting worksites, 
including: (1) the worksites’ Indian status, (2) the worksites’ pre-existing relationship 
with the grantee, (3) the providers’ demonstrated ability to offer meaningful training for 
participants, and (4) participants’ interest in working for specific organizations. Below 
we draw attention to the ways in which these factors influence the worksite selection 
process. 

Indian status. Using Indian organizations as worksites for WEX and CSE 
programs was a deliberate decision for nearly all grantees. Respondents preferred 
using Indian providers because of participants’ desire to work “with their own people” 
who can understand their backgrounds. A number of grantees, both tribal and non- 
tribal, indicated that it was highly important that providers be sensitive to the needs of 
Indians. One grantee selected a service provider based on its efforts to provide 
“cultural sensitivity training” to its staff. One participant remarked at the importance 
of working with other Indians: “It felt good to talk to my own people.” Another 
participant felt a “sense of community and pride” in working for an Indian 
establishment. 

Pre-existing relationship with employers. A number of grantees developed 
strong linkages with a number of departments within tribal governments or other non- 
profit organizations, and in some cases paid particular attention to whether the worksite 
could serve as the permanent employer for WEX and CSE participants. Grantees 
generally selected worksites that had worked with the INA program before and had 
already established a rapport with the JTPA staff. This arrangement facilitated future 
work assignments at the worksite. For example, one grantee located in a metropolitan 
area had developed a strong working relationship with an employment agency and 
frequently collaborates with its staff to help identify and select employers for WEX 
positions. 

Ability to provide occupational training. For some grantees, the provision of 
specific occupational training in WEX and CSE assignments was a significant criterion 
for selecting worksites. While this criterion was not commonly established in our case 
study sites, some grantees specifically selected worksites that can provide occupational 
training. For example, one grantee prefers to assign its WEX and CSE participants in a 
hospital to ensure that participants can be trained in specific occupations on the job, 
such as medical assistance and medical file clerks. In addition, two grantees required 
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worksite supervisors to propose how WEX and CSE assignments can achieve specific 
outcome goals, including a commitment to hire participants upon termination of service 
and the provision of specific training before participants are assigned. Although 
providing specific occupational skills training was identified as important by some 
grantees, we discovered that a majority of WEX and CSE positions provided limited 
training if at all. This is not surprising since a number of grantees view WEX and CSE 
activities as short-term work assignments that offer opportunities for participants to gain 
real work experience. 

Participants’ interests. A number of respondents indicated that worksites were 
selected on the basis of participants’ interests in working at specific organizations, 
especially in tribally controlled organizations or Indian-affiliated agencies. For 
example, one grantee matched a participant interested in working in a clerical capacity 
to a position as a receptionist in a tribal consortium, where she learned about office 
procedures and customer service. Another grantee placed WEX and CSE participants 
at the Indian Health Service on its reservation in response to their strong desire to work 
in a medical facility. 

However, these requests were not always feasible in tribal situations because 
openings were limited. In fact, we found that worksites were not always carefully 
selected to reflect participants’ interests or any of the criteria mentioned above, even 
when the WEX or CSE job assignment was of long duration. Grantees typically 
selected worksites based on the availability of the jobs at a particular worksite at the 
moment. For example, a number of grantees selected worksites when jobs become 
available, as a means to provide any work experience, regardless of the employment 
needs and interests of the participants. One grantee placed the majority of WEX and 
CSE participants in a child care facility, regardless of their career interests and 
backgrounds. Although a number of participants assigned to this worksite expressed 
interest in becoming licensed child care providers, others were assigned because of the 
convenience of assigning participants in a worksite that willingly accepted subsidized 
employment participants. 

Participants Receiving WEX and CSE 

The types of participants enrolled in WEX and CSE services are generally a 
heterogeneous group but they show common characteristics. Many participants, 
particularly those living on reservations, have been long-term unemployed or 
underemployed, lack current skills to compete in the labor market, and have multiple 
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barriers to employment. 3 Many participants reported that, without the opportunity 
offered through WEX and CSE, their prospects for permanent employment would have 
been even more meager. 

In addition to these characteristics, WEX and CSE participants fell into the 
following general categories: 

• Youth participants. A number of grantees indicated that youth 
participants with limited work experience tend to seek WEX and CSE 
services. The demand for services by youth participants meant that 
WEX and CSE positions are often entry-level in nature. 

• Individuals transitioning from public assistance to work. Many grantees 
characterized WEX and CSE participants as those transitioning from 
welfare to work. Since the goals of WEX and CSE are intended to 
introduce participants to the world of work and provide a broad-brush of 
skills, enrolling welfare participants was a common practice. 

• Individuals seeking employment only. The majority of grantees offer 
WEX and CSE services to clients seeking employment rather than 
occupational skills training. Grantees operating in areas with low 
economic growth saw a high demand for immediate jobs rather than 
training. 

Client Selection for WEX and CSE 

The procedures by which grantees select participants for specific WEX and CSE 
activities are unique to the work assignments, and generally include some level of 
employer involvement. Overall, grantees used four main strategies to assign 
participants to work assignments: (1) formal interviews with worksite supervisors, (2) 
informal arrangements between JTPA staff and the worksite, (3) personal relations with 
JTPA staff and/or the worksite, and (4) reverse referrals. 

Among grantees using formal procedures, program staff typically refer several 
participants to employers upon notification of WEX and CSE opportunities. Some 
worksite supervisors then interview potential WEX and CSE participants and select 
individuals according their skill and motivation levels. For example, a worksite 
supervisor in tribal government interviewed several participants for one WEX 
assignment in its Information Services department, in search of participants with good 



3 Barriers to employment included the following: high school dropouts, individuals who are basic 
skills deficient, individuals receiving public assistance, offenders, and homeless individuals. 
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writing skills and experience with customer service. Another worksite, a local hospital, 
views WEX and CSE applicants like job applicants and interviews every potential WEX 
and CSE participant to ensure an appropriate “fit” between the needs of the participants 
and the employers. 

On the other hand, some grantees follow informal procedures in assigning WEX 
and CSE participants and do not require a formal interview with the worksite 
supervisor, especially for work assignments that require minimal work experience and 
skills, such as positions related to general labor. In these instances, grantee staff assign 
participants based on which participants and which WEX or CSE job slots are available 
at the moment. 

In some cases, we discovered that decisions about assigning specific WEX and 
CSE positions are made on the basis of the participants’ personal relationship with the 
staff at the worksite, and, in at least two case study sites, on the basis of tribal 
affiliation. These two tribal grantees gave priority to tribal members when assigning 
WEX and CSE opportunities (but provided opportunities to all Indians seeking other 
employment and training services) . 

Several grantees also practice reverse referrals in assigning participants to specific 
worksites. In these cases, employers selected specific workers they wanted to hire and 
then referred them to the grantee to determine their eligibility for JTPA. For example, 
a participant identified a job opening at Indian Health Services as a medical file clerk 
but the supervisor suggested that she apply to JTPA for WEX or CSE eligibility. This 
employer indicated that it was unable to hire unsubsidized workers immediately and 
needed the assistance of WEX and CSE to support its staffing needs. Although reverse 
referrals are arguably troublesome because many employers would hire participants 
without the support of WEX and CSE services, grantees generally accepted reverse 
referrals because of the dearth of employment opportunities in the local areas and the 
desire to maintain a good working relationship with employers for future placement 
opportunities. 

Content of WEX/CSE 

An assumption of high quality training in our conceptual model, presented in 
Chapter I, states that work-based services, particularly for WEX and CSE, should have 
clear training objectives that promote active learning of work-relevant skills, including 
durable skills, while allowing opportunities for participants to gain evaluative feedback 




on their progress. While many WEX and CSE work assignments offer participants the 
opportunity to acquire workplace skills and broaden their work experiences, some work 
assignments had very modest training objectives. In some cases, this was because they 
functioned as temporary work assignments while imparting some benefit to the 
community. We therefore evaluate these shorter-term work assignments based on the 
coordination of work activities with the needs of the local community. 

Types of WEX and CSE Positions 

Grantees provide a broad-range of entry-level full-time or part-time work 
assignments for WEX and CSE. Work assignments fell into the following general 
categories: 

• Clerical office positions. A number of participants provide overall 
administrative support to worksite staff, including filing, answering 
phones, and data entry. Examples of clerical positions include 
receptionists and file and accounting clerks. 

• Health workers. Grantees also assign participants to work in a medical 
facility at Indian Health Services or local health centers and hospitals. 

Work assignments varied widely depending on openings and 
participants’ qualifications. Examples of these positions include: 
medical assistants, nurse’s aides, general medical intake workers, 
clerical support, certified Emergency Medical Technicians, and staff 
support in an AIDS unit at a community health clinic. 

• General labor. A number of positions related to general labor are 
included in the repertoire of WEX and CSE activities, including general 
maintenance, stock persons, construction workers, and roofers. 

• Social service. Work assignments in the social service sector are also 
offered, including providing staff support in a foster care center, a 
homeless shelter, and at a day care center. 

The wage rate paid for WEX and CSE ranges from minimum wage to $6.00 per hour. 

Training Content for WEX and CSE Programs 

The training content for many of these work assignments varied tremendously. 
Overall, grantees promote modest training objectives for WEX and CSE programs, in 
that these programs are oriented to providing work experience and increasing 
participants’ work maturity skills. Typically, worksites adopt an informal, hands-on 
approach to training, rather than adhering to a formal, pre-established training plan. 
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Although a formal training component is not emphasized for many WEX and 
CSE programs, we found some innovative approaches that were used to provide 
training in a work setting. For example, one grantee develops a contract with the 
worksite and requires all employers interested in accepting WEX or CSE participants to 
submit a written request, along with a job description listing the qualifications for the 
position, work responsibilities for the participant, and responsibilities for the worksite 
supervisor. The contract also highlights training objectives for WEX and CSE, 
indicating areas in which participants are expected to gain competency at the 
termination of the work assignment. Another grantee selects a worksite that provides 
intensive, formal training in Emergency Medical Technician on site as part of its WEX 
assignments. The training consists of CPR, first aid, medical emergency procedures, 
and hands-on training in an ambulance once participants master the fundamental 
concepts of health science. These examples demonstrate the potential for WEX and 
CSE programs to offer formal training as part of the work assignments to assure 
participants gain customized training. 

In other cases, despite the absence of formal training plans, participants expressed 
satisfaction that they were learning important job skills and that their training was self- 
paced with adequate supervision and consisted primarily of “learning by doing. ” 

Entry-level work assignments may also provide valuable training in work 
maturity and basic occupational skills for participants with limited skills or little 
previous work experience, again despite the absence of a formal training plan. For 
example, a 24 year-old single mother with no work experience and multiple barriers to 
employment, including former offender status and basic skills deficiencies, was placed 
in a clerical position at the JTPA office temporarily, where she interacted closely with 
youth. Although her work assignment did not include specific training on the 
mechanics of office procedures, she gained an interest in working in an office 
environment and realized after this experience that she enjoyed working with young 
people and was hired for $9 per hour as a Johnson O’Malley tutor for youth. 

In contrast to these examples, many WEX and CSE positions offered only meager 
training opportunities. Moreover, sometimes striking training mismatches occurred, as 
when participants with sufficient work experience were assigned to entry-level work 
assignments geared towards inexperienced workers. For example, one grantee 
operating in an economically depressed area assigned a participant with extensive work 
experience and training in accounting and business as a stock person in a family service 



center. Although the participant expressed gratitude for the opportunity to work with 
other Indians, he also realized his skills were not being adequately used. 

Unfortunately, at the time of the site visit, the grantee was unable to develop WEX 
opportunities that could effectively meet this participant’s employment needs. 

Intensity and Duration of Training 

The length of time participants enroll in work-based activities can be an indicator 
of the responsiveness of WEX and CSE services. The intensity and duration varied 
significantly depending on participants’ unique circumstances and the grantees’ capacity 
to offer high-quality work-based opportunities. All programs provided WEX and CSE 
services in three categories of duration: high, moderate, and low duration. 

• High duration/long-term work assignments. Participants enroll in WEX 
for 600 to 1 ,000 hours and CSE for the maximum allowable hours 
relative to the grantees’ budget. 4 Participants generally work in a full- 
time capacity for 40 hours per week. 

• Moderate duration. Grantees also provide training at a modest level, 
ranging from 200 to 600 hours. Participants typically worked part-time 
or full-time depending on their needs or the needs of the worksite. 

• Low duration/short-term work assignments. Participants enroll in these 
programs on a part-time or full-time schedule for less than 200 hours, 
typically for several days in between service activities or during their 
job search activities, until they find permanent employment. 

In determining the length of WEX and CSE activities, program staff used several 
criteria, including the needs of the worksite and participants’ interests and service 
needs. Examples of variations in the duration of these services are provided below. 

One grantee provides long WEX assignments of 1,000 hours for some 
participants and “mini” assignments lasting a few days at a time for others. Long 
assignments are provided to individuals with multiple barriers to employment, on 
the grounds that they need the extended work experience to acquire occupational 
skills. “Mini” assignments are designed for those already enrolled in 
occupational skills training and unable to find permanent employment. For 
instance, a WEX participant who has been unable to find a job was enrolled in a 
WEX assignment for a few days to help prepare the mailings for the grantee’s 



4 The statutory requirements restrict WEX services to 1,000 hours maximum and CSE activities to 
10 percent of the annual JTPA budget. 




quarterly newsletter. These brief assignments also provide an opportunity for 
participants deemed not “job ready” to demonstrate their capabilities. 

Another grantee provides WEX and CSE services to individuals undergoing job 
search, and the duration of WEX and CSE is determined by how soon 
participants can find employment. For example, if a participant has not found a 
job quickly, s/he will enroll in WEX or CSE and will remain enrolled until 
finding employment. 

Another grantee prolongs the WEX and CSE experiences by supporting the 
extension of subsidized employment with tribal funds for another 26 weeks at the 
completion of the first 26 weeks of JTPA-sponsored WEX services. Grantees 
providing long-term assignments also explicitly used WEX and CSE programs as 
a placement tool to induce worksite employers to hire participants who have 
engaged in customized work-based training. 

However, sometimes decisions about the duration of WEX and CSE assignments 
were made without careful consideration of either participants’ needs or the trends of 
the local labor market. For example, a grantee located in a high economic growth area 
with a low unemployment rate assigned its job-ready participants in WEX and CSE for 
nearly twelve months. Similarly, many grantees provide WEX services for 26 weeks 
for nearly all of their participants, regardless of their educational and employment 
background , believing that this was the most appropriate decision for participants given 
the poor economic context in which clients live. 

Clearly, grantees face ongoing challenges in developing work assignments that 
can meet the needs of both the participants and reflect the realities of the economic 
conditions in their local areas. 

Quality of Training 

WEX and CSE programs make up a significant portion of case study grantees’ 
service options. Providing these services effectively involves a careful assessment and 
analysis of the needs of participants, employers, and the local community. Ideally, it 
should also be coordinated with local industries and agencies. 

Along these lines, our model of high-quality training presented in Chapter I spells 
out several criteria for effective work-based services. These criteria are especially 
pertinent for WEX and CSE positions of longer duration, rather than those viewed as 
stop-gap assignments (e.g., for those undergoing job search and who are expected to 
secure unsubsidized employment relatively quickly). These features include: 
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• Clear training objectives. Work-based activities should have clear 
training objectives so that participants can acquire new, durable, and 
transferable skills. 

• Good match between participants ’ needs and work assignments. Work 
assignments should parallel participants’ training needs, interests, and 
skill levels. 

• Effective supervision and monitoring of participants ’ progress. 

Supervision of worksite activities should allow opportunities for 
participants to gain evaluative feedback and promote active learning and 
to ensure effective supervision and mentoring. 

• Coordination of work-based activities with community initiatives and 
programs. When work-based services are used to improve occupational 
skills, grantees should demonstrate a strong coordination of work 
activities with other initiatives in the local community. 

In implementing WEX and CSE according to these guidelines, grantees faced 
challenges due to internal and external constraints, including limited funding and 
limited employers to provide WEX and CSE opportunities. 

Developing Clear Training Objectives 

To be most effective as a training vehicle, WEX and CSE services should have 
clear training objectives that can present opportunities to gain new skills. On the 
whole, many grantees emphasized that WEX and CSE activities should address work 
maturity skills and, sometimes as a result, specific training objectives were poorly 
specified. One grantee, for example, emphasized the importance of “getting 
participants over the fear of what it means to work. ” Similarly, a number of grantees 
identified generic training objectives for WEX and CSE programs, such as using WEX 
and CSE “to enhance the employability of individuals through the development of good 
work habits and basic work skills.” 

Overall, we found only modest training objectives for WEX and CSE services. 

In fact, training objectives were not clearly developed in most case study sites. 

However, several grantees were the exception and demonstrated a concern towards 
obtaining specific training goals through WEX and CSE. An example of a grantee 
establishing specified training objectives is presented below: 

One grantee established a clear training policy and developed a training contract 

similar to an OJT contract for all of its WEX and CSE assignments. The grantee 

and the worksite supervisor at the children’s shelter jointly develop a training plan 

that outlines specific competencies to be gained at the completion of the work 
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assignment. Several objectives were identified, including (1) attaining clerical 
skills in record keeping, filing, and answering phones; (2) participating in project 
outreach by corresponding with youth clients and their families; and (3) serving 
as a co-trainer to peers involved in peer counseling. The contract also outlines 
responsibilities for the worksite supervisor and the participant. 

In some cases, the presence of training goals was due to the efforts of the 
worksite rather than the policies established by the grantee. For example, a worksite 
supervisor in the tribal enrollment department developed several learning goals for the 
work assignment, including: understanding the computer systems used by tribal offices; 
understanding tribal enrollment procedures such as indexing, filing, and searching the 
database; gaining proficiency in clerical skills; and understanding how different tribal 
offices interface with one another. 

Matching Participants to Work Assignments 

Grantees faced challenges in their efforts to match participants to appropriate 
work assignments. As previously mentioned, grantees were often constrained by 
several factors in matching participants to work assignments, including: (1) limited 
work assignments in their local areas, (2) participants’ need for immediate income, (3) 
transportation challenges in remote areas, and (4) the needs of the tribe, which 
sometimes determined the kinds of work assignments available to participants. 

Partly because of these constraints, we observed that many work assignments 
under WEX and CSE, even those of long duration, were not related to participants’ 
career paths. Nonetheless, we identified several exceptions earlier in this chapter that 
demonstrated that some grantees pay very careful attention to the matching process. 

Supervising WEX and CSE Participants 

Our model of high quality training suggests that supervising participants should 
occur in the form of mentoring and coaching rather than “teaching,” and provide 
opportunities to learn through evaluative feedback on participants’ progress at the work 
place. We found that worksite supervisors played a large role in monitoring 
participants’ progress and grantee staff exhibited varying levels of effort to document 
participants’ learning gains. 

We observed that worksite supervisors held high expectations of WEX and CSE 
participants and treated WEX and CSE participants “like other employees,” and 
expected them to work “just as hard” as non-WEX and CSE staff persons. Below we 
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highlight examples of innovative strategies grantees and worksite supervisors 
implemented to supervise and mentor WEX and CSE participants. 

One worksite requires WEX and CSE participants to get written evaluative 
feedback on a monthly basis for the first two months, followed by an evaluation 
after every three months. The performance evaluation lists statements (e.g. 
“Employee is consistently punctual”) and rates participants’ performance on a 
four-point scale: excellent, good, fair, and poor. 

In another case study site, all participants receive feedback on their performance 
on a weekly basis or depending on the schedules of the worksite supervisors. 
Although the grantee enrolls WEX and CSE participants at this worksite on a 
regular basis, the grantee only enrolls one participant in one department at a time, 
allowing opportunities for participants to receive individualized attention from the 
worksite supervisor. 

Training at one case study site was highly interactive. Instructors and day care 
staff serve as “mentors” and provide individualized instruction to participants. 
Staff and participants “work side by side” at the day care center, and staff 
provide supervision when necessary. Staff at the day care center model ways to 
handle infants and ways to guide activities with pre-schoolers and afterschool 
children. Participants later simulate the same activities. Participants are 
continually observed on how they adapt to their assigned tasks and are given 
feedback of their performance. 

Some participants indicated that they were allowed a high-level of autonomy in 
their work tasks while others were closely supervised. Supervisors reported that in 
general, they found participants to be hard working, skillful, and reliable. 

Some grantee staff also facilitated mentoring and coaching by documenting 
participants’ progress and ensuring that supervision was occurring at the worksite. 
Typically, grantees scheduled monthly meetings with worksite supervisors to discuss 
concerns about participants’ performance and other issues that may have arisen during 
the course of the training. A number of grantees also telephoned the worksite once a 
week to discuss how participants adapt to their roles. We also found evidence to 
suggest that a number of grantee staff played a limited role in mentoring participants 
during their training in WEX and CSE because of high case loads and limited human 
resource capacity. 

Individualized Attention 

Another important dimension to learning is the amount of individualized attention 
participants receive at their worksite. Nearly all case study sites provided 
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individualized attention to participants , and the ratio between supervisor to participant 
is low, typically ranging from 1:1 or 1:5. 

Worksites promoted several approaches to facilitate individualized instruction. 
Typically, worksites assign a permanent staff member, (e.g. a secretary, or a 
department supervisor) to provide individualized training at the worksite. Grantees also 
refer one participant to worksites at a time to ensure that the experience provides 
opportunities to learn on an individual basis. Each of these approaches enabled 
supervisors to fine-tune the instruction to match the participants’ learning pace. In one 
worksite, a supervisor assigns “easy” tasks first before progressing to more complex 
tasks, and closely observes the participant to measure the participant’s proficiency 
before assigning new tasks. 

Coordination of Work Activities 

An important objective of WEX and CSE services is to contribute to the overall 
economic and community development of Indian communities. A number of grantees 
coordinate WEX and CSE services with other community initiatives to expand the 
resources available to participants and their families. A number of grantees leverage 
existing resources such as tribal funds and other state funds to continue to support WEX 
and CSE participants. For example, several tribal grantees arranged for the tribe to 
increase the wage rate of WEX and CSE above the minimum wage and extend the work 
assignments so participants can further gain training at the worksite. 

Conclusion 

This chapter provided an in-depth analysis of WEX and CSE services in 23 case 
study sites. We found that these service activities can meet very diverse needs and 
serve many objectives, including, from the participants’ standpoint: teaching 
occupational skills training, providing those with limited work histories an exposure to 
the work world, boosting self-esteem and self-confidence, teaching pre-employment and 
work maturity skills, providing immediate income, and offering an entree to permanent 
employment in positions with the tribe for which the competition is fierce. From the 
standpoint of the community, the subsidized labor associated with WEX and CSE help 
tribal and other public agencies provide community services that would otherwise be 
prohibitively costly. 

It is surely a challenge for grantees to design positions that balance these diverse 
and often competing objectives. One way they do so is by varying the duration of the 
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WEX and CSE assignments. Thus, positions sometimes last just a few days, as when 
they serve primarily as stop-gap measures to provide immediate income for participants 
who are undergoing job search. In these cases, training objectives are typically modest 
and are rarely well-specified, as seems appropriate given the nature of the assignment. 
At the other extreme, WEX placements last up to the 1,000-hour allowable limit, and 
CSE is used to prolong the assignment still further. These examples pose more difficult 
dilemmas for grantees, as they must meet the employing agencies need for labor in 
specified jobs, meet the participants’ short-term need for immediate income and work 
experience, and meet their longer-term need for skill training that will promote long- 
term employability. Grantees balance these multiple objectives with mixed success. 



I 
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IX. TRAINING ASSISTANCE AND PLACEMENT 
AND POST-TERMINATION SERVICES 



This chapter describes the services associated with the Training Assistance, 
placement, and post-placement components of the JTPA INA program among the 23 
grantees participating in our study. The services are crucial to the programs and to 
program participants because they largely determine the degree to which prior program 
investments enable individuals to transition into careers leading to self-sufficiency. 
Moreover, these services reach the largest number of participants, compared to other 
service activities, and thus have the potential to make an enormous impact on the 
economic and social health of the communities served by INA programs. 

This chapter will discuss findings resulting from visits to our 23 sampled grantees 
and in-depth observations of training assistance (TA) services in four sites. First, we 
provide an overview of the TA service component. Next, we discuss the objectives of 
grantees in providing TA services to their participants. Finally, we address the design 
and delivery of these services, pointing to specific challenges the grantees encountered 
and innovations they developed. 

Overview 

Federal legislation guiding the implementation of JTPA INA programs identifies a 
variety of services that may be provided under the Training Assistance component. 
These include: 

• Orientation to the World of Work— including training designed to 
prepare participants for the demands of full- or part-time employment. 

• Counseling— including employment-related testing and job counseling. 

• Case management— including monitoring the clients’ progress through 
training or other services and serving as a facilitator. 

• Job development— including marketing placement services to employers. 

• Job search assistance— including job search skills instruction, individual 
job search or career planning, and other specialized services leading to 
unsubsidized employment on the part of participants. 

• Job referral and placement— including matching individual participants 
to jobs for which they are qualified. 
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• Vocational exploration — including providing special activities designed 
to assist participants in making informed career choices. 

The objective of these services is to assist participants in moving into 
unsubsidized employment situations by linking them to quality jobs or information 
about jobs for which they may be qualified. Most grantees have also realized the 
importance of providing participants with job-search training under this service activity. 

All grantees in our sample provided TA and related services to their participants. 
We were therefore able to conduct interviews about TA services with program staff and 
participants at all sampled sites. At four of these sites, we were able to observe service 
activities in progress, and conduct interviews with participants, program staff, case 
managers, and, in cases where TA services were provided by contractors, the service 
providers as well. We also reviewed the case files of participants who had received TA 
as a part of the total service packages, and those of participants who had received solely 
TA services. 

Although all sampled grantees provided TA services, the emphasis they placed on 
it relative to other services, and the depth and breadth of those services, varied 
enormously. In some cases, strategic decisions resulted in TA services as an area of 
emphasis; in others cases, an emphasis on other service activities resulted instead. 

For example, we used the percentage of program participants receiving only T A 
(available from the Annual Status Reports) as an indicator of the level of importance 
placed upon direct placement services, which represents an important component of 
TA. 1 These percentages among our sampled grantees ranged from 1.2% to 95.6%. 

The following table illustrates this variation. 



1 This reporting item represents those participants who, during the program year, received 
Training Assistance and no other training or subsidized employment, at the time the ASR was submitted. 

2 Two sampled grantees did not report figures for this variable. 



Pet of participants 
receiving only TA services 



Number of 
Sampled Grantees 



0%-25% 

25.196- 5096 

50.196- 91.596 



11 

5 

5 

21 total grantees 2 
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Objectives of TA and Related Services 

Grantees providing TA services in urban, higher growth areas faced somewhat 
different challenges than those serving participants in reservation areas where jobs were 
scarce. For example, in urban areas, job opportunities were often much more plentiful, 
but grantees need to attend to some participants’ lack of preparation for urban life and 
their sometimes unrealistic work- or income-related expectations, along with the 
absence of family or community networks offering them support. These grantees also 
needed to overcome the bias of employers who were less willing to "take a chance" on 
Native American participants. Urban programs designed their services with these 
challenges in mind. 

By contrast, in reservation areas, job opportunities— especially private sector 
jobs— were typically hard to come by or quite limited in the range of occupations or 
work settings that were available. Moreover, many tribal programs were providing 
services in areas with low population density, making it difficult to achieve economies 
of scale in providing any services associated with the JTPA program, TA services 
included. As a result, the service designs and delivery mechanisms associated with 
these programs were quite different from those of their urban counterparts. 

Despite these important differences, most grantees perceived TA services as 
important and identified several common objectives of the TA components of their 
programs including: (1) providing general guidance and counseling, (2) assisting 
participants in identifying career interests and exploring potential career options, (3) 
assisting participants in developing plans to achieve their career goals, (4) assisting 
participants in transitioning from another service activity into unsubsidized 
employment, (5) assisting participants in securing unsubsidized employment, and (6) 
building the capacity of individuals to achieve self-sufficiency and promote healthy and 
prosperous Indian communities. Each of these is addressed below. 

• Providing guidance, support, and counseling to program participants as 
needed. Many grantees provide counseling to participants not just to 
support career guidance but on a whole range of personal, family, and 
community issues, as a part of the comprehensive service approach 
common to these grantees. Given the enormous health and 
socioeconomic challenges confronting the service population, this 
service is crucial in helping participants prepare for participation in 
other service activities as well as for employment. 

• Assisting participants in identifying career interests and employment 
options. Grantees enroll many participants who have never held jobs or 
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who have not been exposed to the world of work on a regular basis. As 
a result, helping participants explore their own interests before enrolling 
them in a specific training or work experience program is often an 
important component of training assistance. 

• Assisting participants in establishing career (and personal) goals and 
developing plans to achieve them. Because career planning for a Native 
American clientele often involves difficult family decisions (such as 
whether to relocate where there are more opportunities or compromise 
career interests to remain near family and community), grantees’ ability 
to help participants establish realistic and consistent career and personal 
goals is crucial in achieving high-quality participant, community, and 
program outcomes. 

• Assisting participants in transitioning from another service activity into 
unsubsidized employment. Grantee staff often use training assistance to 
facilitate the transition from subsidized employment or classroom 
training into unsubsidized jobs. This can be accomplished through job 
development activities or by providing instruction and assistance to 
participants in individual job search strategies. 

• Assisting participants in securing unsubsidized employment. Grantees 
also use training assistance to facilitate participants’ direct entry into 
unsubsidized employment, either through job search training, job leads, 
or other supports (e.g., supportive services). 

• Building the capacity of individuals to achieve self-sufficiency and 
promote healthy communities. For many grantees, providing training in 
job search and in navigating public and private agencies and programs 
for additional supports, or for jobs, was an important part of training 
assistance activities. These services enabled participants to engage in 
job search not only immediately, but over time as their career interests 
and needs changed. Several grantees also noted that many participants 
who had achieved self-sufficiency served as valuable family and 
community role models. Additionally, particularly in urban areas, 
grantees place great emphasis on creating community (rather than 
working within it, as in reservation areas), as part of the TA activities. 
Participants indicated that this is a key service provided by INA 
grantees in urban areas. 

Designing and Delivering Services 

Important aspects of the design and delivery of TA services in the programs we 
studied included promoting access to services and ensuring that services were delivered 
in a way that was sensitive to participants’ and communities’ cultural values. 




Promoting Access to Services 

INA grantees spend a great deal of face-to-face and one-on-one (over the 
telephone or in-person) time with their customers. Common one-on-one activities 
included job-search assistance, career counseling, and "problem-solving." As noted 
earlier, a particular challenge INA programs face is the frequency and intensity of 
significant needs in addition to work among their customers. As a result, staff typically 
stretch "career counseling" into general counseling on a wide range of issues, including 
mental and physical health, parenting, child-care, and transportation. 

One challenge faced by grantees, however, was that of promoting access to these 
counseling services. As was discussed in Chapter V, customers with good personal or 
familial networks or those residing closest to service delivery points had an easier time 
accessing services than individuals who did not meet these criteria. However, far-away 
customers were no less in need of services, prompting grantees to identify multiple 
ways to reach participants, in the face of dispersed service areas and, in many cases, 
limited transportation networks. Because training assistance was such a commonly 
provided service activity, strategies to promote access were particularly important to 
ensure that customers got the career guidance and other placement assistance they 
needed. Among the strategies grantees used were: (1) combining and sequencing 
training-assistance services for the convenience of customers, (2) contracting services 
rather than providing them in-house, using outstations, or establishing joint referral 
networks, (3) subsidizing or actively taking responsibility for customers' transportation 
needs, and, to a lesser degree, (4) promoting self-access services. 

Combining and Sequencing Services. Grantees facing transportation (and 
related) challenges tended to combine and sequence training assistance in a variety of 
ways. For example, some grantees employed creative scheduling to concentrate 
services in as few visits as possible, so that participants were not forced to visit the 
service site as frequently as they might have otherwise. Thus, several grantees 
arranged for customers to meet with their counselors right after their orientation 
program or right before an interview. This way, customers could access multiple 
services during the same pre-scheduled visit. Similarly, programs might introduce 
career exploration as an activity, and then encourage participants to engage in it 
immediately. In this way, customers could generate the highest added value for each 
trip they made to the service site. 




Contracting Services, Outstationing Staff, or Forging Joint Referrals. 

Another way grantee staff attempted to make services easy for participants to access 
was to contract services to private or public providers. This strategy helped make 
services more accessible to customers. For example, grantees may not have provided a 
"World of Work" seminar in-house, but they required it of their contractors for CRT, 
whether basic or vocational education. This way, providers of CRT might devote 
several class periods to work/life-skills as a part of the overall training program. 

As discussed in Chapter V, many grantees also established satellite offices. 
However, several indicated that budget cuts over the years had forced them to reduce 
the number of satellite facilities they operated. For example, one grantee had closed 
two "satellite" offices as a result of such budget reductions. As an alternative, some 
grantees regularly rotated service sites by outstationing staff at strategic locations in the 
communities they served. Thus, customers may meet their career counselor/case 
manager for the first time in the main service site (e.g. JTPA program headquarters), 
and then meet them in a site closer to their home on subsequent occasions. Associated 
with this strategy, some grantees assigned case managers to customers based on 
geographic proximity, so that it was possible for staff to provide services from remote 
locations that were near multiple customers and near staffpersons’ places of residence. 

As an alternative strategy, staff might spend several days per week in their 
"home" office, and the rest of their time providing services out of the offices of partner 
agencies. For example, one grantee provided services out of a community college 
several days per week, while another provided services through a state agency office. 

Some grantees also established joint referral networks or cooperative agreements 
with other social service agencies. For the most part, however, the establishment of 
partnerships with other agencies that would enhance the ability of grantees to extend 
their service reach was an underutilized strategy, particularly for urban sites. For 
example, while many grantees had developed high-quality partnerships with local 
employers or other local service providers, few had more than cursory relationships 
with traditional providers of workforce development services, including those leading 
the federal and state One-Stop initiatives. Although some grantees expressed reluctance 
about establishing partnerships with non-Indian providers or providers outside their 
immediate areas, linkages to organizations and agencies involved in One-Stop have 
tremendous potential to connect grantees to resources, training opportunities, and 
technology at minimal cost to the grantees. Other grantees had the impression that 
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One- Stop was an “urban” initiative. However, one of the capacities that One-Stop has 
helped promote is long-distance learning, a service that could greatly benefit customers 
of tribal JTPA programs (and in many cases, is already benefiting customers connected 
to community colleges and other public and private training providers). Moreover, the 
one grantee that had developed high-quality partnerships with the state and local 
organizations and agencies involved in One-Stop was a rural grantee. 

Still other grantees either did not know about the One-Stop initiative or did not 
wish to participate in the partnerships developing to support it. In some cases, 
however, this was clearly a missed opportunity. One grantee, for example, was 
attempting to provide comprehensive services (on a shrinking budget) in an urban area 
with a plethora of good job opportunities. Grantee staff were engaged in labor- 
intensive one-on-one job matching activities among small employers that were generally 
not connected to the One-Stop systems in their areas and that paid low wages. In 
essence, the grantee was inadvertently competing with providers with more resources 
and better service networks and, as a result, undermining its capacity to provide the 
specialized supportive services its participants needed most. In other words staff were 
attempting to duplicate a comprehensive service system rather than emphasizing the 
unique supports that they could provide better than their competitors. 

In contrast, another grantee has witnessed numerous benefits as a result of its 
participation in the state's One-Stop initiative. This site was engaged in region-wide 
strategic planning around economic and workforce development— largely as a result of 
the partnerships formed through One-Stop. Partners held monthly planning meetings, 
alternating sites. One of these meetings took place during our site visit. Local partners 
representing the three surrounding cities met at a tribal center to plan school-to-career 
events on a regional basis. 

We hasten to add that One-Stops should not be viewed as substitutable for the 
services that are provided by specifically Native American grantees to their clients— as 
we have indicated in a number of places throughout this report, IN A grantees serve a 
unique and irreplaceable role in meeting the needs of Indian people and communities. 
However, the partnerships forming in response to the federal and state One-Stop 
initiatives and the support for collaboration provide valuable opportunities for urban 
and rural grantees alike to gain access to additional resources. These are opportunities 
that should not be missed. 
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Linking Customers to Transportation Resources. Many grantees made access 
to transportation or information about transportation a key part of their overall menu of 
services. This occurred in several ways. In some cases, grantees directly reimbursed 
customers for mileage or played an active role in promoting ride-share programs— 
programs that link individuals with common transportation patterns to one another. 
However, this was not an option for many customers who did not have access to private 
transportation at all. 

In such cases, program staff provided tokens or passes on public or available 
private transportation (e.g. employer-sponsored shuttles) and advocated on behalf of 
their customers for increased access to public transportation. Again, however, these 
services did not meet the needs of individuals who did not live near public 
transportation lines or were not yet employed or otherwise connected to a potential 
shuttle service. 

In some cases, grantees themselves sponsored a shuttle that would take 
individuals to common points of interest, such as schools, large employers, public 
transit routes, or to the grantees’ place of service. Although the shuttles offered 
transportation alternatives to customers with no access, they served not just the JTPA 
programs, but multiple programs sponsored by the grantee. As a result, they were not 
wholly reliable, making it difficult for customers to arrive at their destinations in a 
predictable manner. 

None of these options "solved" the transportation challenge, but they all served as 
support that JTPA INA participants could access. 

Self-Access Services. Although self-access services stand out as a key means of 
extending access to participants and eligible customers for whom traditional service 
delivery methods may not be adequate, grantees tended not to emphasize self-access 
services in their programs. 3 There were a variety of reasons offered for this including: 
(1) the potential for participants to be intimidated by self-access services; (2) the lack of 
available technology; and (3) disinterest on the part of staff or participants (and, 
conversely, the high emphasis placed on personalized face-to-face services). 



3 Few programs, whether rural or urban provided self-access services. 
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While each of these explanations has its validity, self-access services may be 
underestimated as a tool from which INA grantees may benefit, especially in providing 
training assistance services. For Indian employment and training programs, the number 
and variety of workforce-related resources available through the Internet is mind- 
boggling. Moreover, as a result of enormous investments by U.S. government 
agencies, such as the Department of Labor (including DINAP) and the Department of 
Education, many of these resources are in the public domain and can be accessed 
without a fee. These resources include tools to educate customers about the types of 
careers available and what is required to pursue them, tools to assist customers in skill- 
building, tools to assist customers in accessing information about jobs, and tools to 
assist customers in accessing the information they need to make informed job/career 
decisions. 

Perhaps even more important is the on-line information (and resources) available 
to assist staff in linking their customers to information and services that meet their 
needs — high-quality and locally-relevant labor market information, for example, is 
difficult to access without technology. Moreover, the communication that electronic 
connectivity makes possible may enable staff to better advocate on behalf of their 
customers with other programs, service providers, or decision-making bodies 
(government or tribal representatives) . 

Moreover, for rural sites in particular, electronic information about careers that 
are not available locally may be the only information customers can use to make their 
career choices. Thus, in this context technology has a role to play in connecting 
participants to opportunities that may exist for jobs elsewhere (where there are no 
alternatives or the customers choose to leave their areas for a time), home-based 
enterprises, temporary (off-reservation) work or apprenticeships, short-term non-local 
educational opportunities, or small-business assistance. Additionally, and perhaps more 
importantly, technology can give Indian (or non-Indian) employers a way to find 
qualified trainees, employees, or apprentices enrolled in JTPA programs — locally or 
long-distance. 

Both INA grantees offering self-access services, and other programs serving 
special needs customers, can lend insight into the challenge of making appropriate and 
value-added self-access services available to customers of INA programs. For 
example, although some customers may indeed be intimidated by self-access services, 
the experience of some programs is that the learning curve among customers is often 
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very quick and very steep. One rural grantee had begun to use long-distance learning 
in its vocational and community college programs with great success, and was 
exploring the possibility of providing other work-related training to long-distance 
learners using this technology. Another grantee had developed a resource room 
consistent with the state's standards of self-access services and found that participants 
took to it quite well. 

These efforts, most of which were new and not yet fully operational, nonetheless 
witnessed success in achieving their planned objectives and led to additional positive 
unintended outcomes as well. One program reported that, as customers became more 
familiar with technology, they could better assist their children with homework. 

Another reported that with technology training, their customers had begun to 
demonstrate more confidence in their interactions with all agencies, service providers, 
and commercial exchanges because they had a greater sense of control over their 
situations . 4 

While self-access services cannot and should not replace face-to-face contact, they 
can help programs provide better services to more customers. In particular, technology 
can assist in the provision of a variety of training assistance services— orientation, 
counseling, job-development, job-search, placement, and vocational exploration— and 
can enable skill-building at the same time. At the same time, it must be acknowledged 
that the effective use of technology requires a substantial up-front effort that will be 
difficult for many grantees to make. 

Delivering Culturally-relevant Services 

Because training assistance activities are among the most widely accessed, they 
represent a particularly important vehicle for delivering services that meet the special 
needs of the Native American job-seeking populations. Along these lines, we met with 
staff that indicated that their organizations should be serving as "practice grounds" for 
individuals preparing to enter the workforce or enroll in training programs. This might 
mean enforcing "appropriate" workplace norms and behaviors with consequences for 
non-compliance. Alternatively, it might mean creating a safe environment in which 
participants are free to make mistakes without fear of reprisal so that they could serve 



4 A specific instance noted was increased comfort in using debit card transactions at the grocery 

store. 
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as learning opportunities for all. Or it might mean simply providing a work 
environment and work experience for individuals preparing for subsequent jobs or jobs 
in non-Indian environments. 

Most grantees, however, did identify at least five interrelated features of their 
programs that either made them different from non-Indian employment and training 
programs or made their programs stand out as crucial services in the Indian 
communities they served. These included: (1) placing the family (and extended 
families and communities) at the center of services rather than the individual 
participant; (2) identifying appropriate employment opportunities that meet the unique 
needs of participants; (3) connecting Native American job seekers with Native 
American employers; (4) ensuring that non-Indian employers have access to skilled 
Native American workers (e.g., on reservations, by connecting the TERO to the 
employment and training systems); (5) serving as a bridge, whether through 
information or work experience, between Indian participants and work and learning 
opportunities in the non-Indian economy; and (6) providing some level of personalized 
post-placement supports to increase job-retention and advancement. 

Placing Family and Community at the Center of Services. Increasing numbers 
of providers of employment and training services are discovering that an individual's 
employment needs cannot be met independently of other basic needs, such as mental 
and physical health, self-esteem issues, and the more tangible barriers of transportation 
and child-care. They call this addressing the needs of “the whole person.” Providers 
of JTPA INA services have a long history of adopting this approach and typically take 
it even one step further by helping participants care for the “whole family.” In 
practice, this means that providers attempt to help participants address employment 
barriers that lie far outside of JTPA boundaries by connecting them to a wide range of 
other services and supports provided by the tribe or in the community. For example, if 
a participant is caring for an elder relative, and therefore not able to work, a JTPA case 
manager may bring in elder care services, or help the participant find alternative ways 
to meet that need, prior to providing job-related services to the JTPA participant. 
Alternatively, when family situations change as a result of a working parent, case 
managers may play a role in helping the family identify new activity patterns that 
support the family in its new situation. 

Meeting families' comprehensive needs can potentially have a critical impact on 
the effectiveness of employment and training services. To their great credit, all of the 
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programs we visited to some degree had integrated an element of “family care” into 
their menu of services. This constituted one of their key strengths. 

Identifying Appropriate Work Opportunities. Another role that the JTPA staff 
can and do play is identifying work opportunities that meet the unique needs of the 
participants. This role was a difficult one for grantee staff to navigate. On the one 
hand, grantee staff seek to assist their customers in identifying employment 
opportunities in which they are likely to be successful. As a result, the programs tend 
to provide a great deal of support in the way of identifying appropriate job leads. 
However, part of the objective in providing job search assistance is to help participants 
develop job search skills, thereby empowering the participants to seek and find their 
own jobs in the future. This kind of assistance/instruction was less common than 
wholly assisted job search. In other words, grantee staff were more likely to provide 
more placement assistance than job search assistance, leaving the participant dependent 
upon assisted services again the next time a job search was necessary. 

Several pitfalls emerged as a result of emphasizing placement assistance over job 
search skill-building activities. First, as case managers identified opportunities that 
they felt would be appropriate for their participants, they risked playing more of an 
advocacy role for their participants than providing placement or matching services for 
their employer customers. As a result, some programs were perceived by employers as 
"difficult to work with" or as referring job seekers who were ill-prepared. 

Second, when programs did not promote job-search skills acquisition, they often 
found themselves serving many repeat customers. For example, because staff often 
(out of necessity) referred customers to seasonal jobs rather than permanent ones, 
customers needed services again within a short time frame. In their turn, because these 
customers had used JTPA services already and were familiar with them, they returned 
as they felt they needed further assistance. However, these are the very customers who 
should be seeking lower levels of assistance each time they return. Instead, for the 
most part, these customers were re-enrolled in the program and were again provided 
with very similar services as they had been given the first time around. Not only does 
this practice concentrate limited resources on fewer numbers of participants than would 
be possible with increased emphasis on job search skills, but it also places participants 
in a position of dependence vis a vis program staff. 




As an alternative, which some grantees demonstrated quite well, program staff 
can exercise more diligence in working with employers to uncover healthy employment 
situations for participants without compromising participants’ opportunities to learn job- 
search skills. As an example of this: 

One program focused its job search assistance activities on "professional 
development" or career advancement instruction. Participants were encouraged 
to work with their employers to identify the tasks they must perform as a part of 
their jobs, and then develop measures of quality so that the participants knew 
when they were doing well. The program next provided instruction in "asking 
for training and other advancement opportunities" so that the workers could 
advance their careers. Three of the four participants we interviewed at this site 
had directly benefited from this instruction within months of starting their new 
jobs. Program staff indicated that providing such instruction up-front led to 
steady career advancement and prevented participants from returning for services 
they really did not need. 

Matching Job Seekers with Native American Employers. In their efforts to 
create "sheltered” work experiences that meet participants' needs, programs tend to 
concentrate their efforts on employment opportunities among tribal agencies or 
departments or for (the limited number of) Indian-owned businesses. This matching 
can result in a rewarding and valuable experience for both employer and participant. 
Moreover, most programs made it part of their mission to meet the needs of Indian 
employers as a part of doing business, and providing such firms with a source of skilled 
workers very nicely served this function. Similarly, participants often express a strong 
preference for working in an Indian-owned firm, so their needs are being advanced as 
well. Since every urban area we visited was experiencing high levels of economic 
growth and low levels of unemployment, grantees found that matching participants with 
Native American employers was easier than it had been at any time in recent memory. 

However, the more successful grantees discovered that there are several pitfalls to 
avoid in pursuing this strategy. Among these pitfalls, we encountered several situations 
in which tribal agencies or departments came to perceive that they were "owed" a 
particular number of participants in subsidized employment each year. Grantees 
sometimes obliged these agencies by sending a steady stream of participants, regardless 
of the work available or the skills or interests of the participants. 

Similarly, grantees must be careful to strike a balance between the desire to place 
participants in Native American controlled work settings with the need to provide 
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positions that pay well, offer good benefits and opportunities for advancement, and 
meet participants’ other needs (e.g., convenience, ready access, etc.). 

At one extreme, one program in an urban area was attempting to place all of its 
participants with Native American employers, and thus made almost no referrals to 
non-Indian businesses. As a consequence, participants were not made aware of good 
opportunities with non-Indian employers and did not have the chance to weigh the pros 
and cons of the alternatives for themselves. To take another example, a grantee located 
near a shopping mall offering wages in excess of $9 per hour referred a participant to 
an Indian-owned firm across town that required her to travel on two different buses 
each way for a wage of $5.50 per hour for a position that was physically demanding. 
While these represent extreme examples, they do speak to the larger and complex 
question of grantees’ obligation to balance the needs of the participants with the needs 
of the employers, and the participants’ short-term with their longer-term objectives, 
while helping participants make informed choices. 

Matching Native American Job-seekers with non-Native American 
Employers. Another important theme identified by the grantees we visited was that of 
assisting participants in preparing for and transitioning to non-Indian employment 
environments. Like some of the services addressed above, these transition services 
must be navigated carefully. We found several successful, although not particularly 
institutionalized examples of such services that appear to be quite valuable to 
participants. In one case, much of the transition- related services took place during one- 
on-one counseling sessions between case manager and participant. In another case, a 
service provider took groups of participants seeking employment in urban areas on 
“field trips” to see what the work environments were like and meet with employees 
working in them. 

Often, though, these services, however greatly needed, can “fall through the 
cracks,” leaving participants unprepared for their jobs in their new environments. This 
can have negative repercussions beyond the individual cases. One employer contacted 
during a site visit indicated a reluctance to rely too heavily on the JTPA program for 
hiring assistance because former participants were not always prepared for the work or 
life challenges that awaited them. 

Providing Post-placement Service and Supports. An extension of transition 
services, post-placement support is another valuable service provided by some grantees. 




This service is potentially very important, because, due to limited job and career 
choices, many participants may settle for positions that are available rather than those in 
which they have a genuine interest. Case managers can play a valuable role in assisting 
participants in adapting to their new positions and in identifying their advantages (e.g., 
new skills they are likely to develop that will help them acquire more desirable 
positions over time) . 

The strategies used by grantees to provide post-placement support included 
periodic telephone “check-ins,” using the moccasin telegraph to inquire (informally) 
about the status of participants, contacting the employer to discuss participants’ 
progress, and visiting work-sites to insure participants are engaged in their work. 
Although we did not encounter them, we also heard about programs that matched 
participants with employers in a formal mentoring arrangement. 

Although JPTA funds can be used to support post-placement activities, lack of 
funding was the most significant challenge faced by grantees in providing these 
services. Moreover, regulatory requirements do limit post-placement services to no 
more than 90 days after termination has occurred. 

Conclusion 

This chapter provided insight into the strengths and challenges of the TA services 
among the 23 grantees participating in this project. The programs, whether rural or 
urban, faced clear challenges including (1) identifying and building partnerships that 
enhance services and access to them, and (2) promoting empowerment as a theme 
within the programs through self-access services and technologies. 

Overall, the ability of staff to exercise the whole person/whole family approach in 
designing service strategies for participants was a clear strength. Rather that attempting 
to “fix” participants, staff of INA programs exerted great effort to meet the immediate 
needs of participants in an effort to support the objective of livable wage employment. 
Participants appreciated this approach and felt that it was unique to JTPA INA 
programs. 

As the world of employment and training/workforce development continues to 
change, Indian grantees face an opportunity to develop some key service strengths and 
market niches. One of these niches that appears to be in demand among customers may 
be the ability to help participants transition into a diverse workforce. Although many 
grantees attempt to smooth these transitions, in most cases, grantee staff set-up a 
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white/Indian dichotomy to demonstrate differences. However, Indians who leave their 
reservations, and even those who do not, will increasingly encounter a more diverse 
workplace in which the rules (once perceived as white) will continue to change. 
Participants may need to be prepared to take advantage of these new situations. 

Moreover, as employers, with ever-diverse markets, seek to diversify their 
workforces to better reflect their customers, Indian grantees, particularly urban 
grantees, may be in a good position to help their participants benefit from these trends. 



X. INFLUENCES AND EFFECTIVENESS 



Tribes and Indian organizations are enabled to receive funds from DOL by action 
of the U.S. Government based on the treaty relations that tribes have with the federal 
government. This concept was expanded upon in recognition of other federal laws and 
policies that established reservations; dissolved some reservations; relocated tribal 
people from reservations; provided education on and off the reservations; and then 
provided numerous federally-funded programs and services in areas where tribal people 
reside. The federal government was/is attempting to meet the treaty obligations by in 
essence "... providing health, education and welfare ... as long as the grass grows and 
water flows”— in other words, in perpetuity. These services are provided in exchange 
for the various concessions agreed upon by the federal government and the tribes (land, 
acquisition, peace, etc.). Special laws, attending rules, and regulations have been 
passed and amended to address the special, unique status of tribes and tribal people. 
Accompanying these congressional acts, funds were budgeted and appropriated. 

Along these lines, this report has examined how funds have been used to provide 
employment and training services to Indians and Native Americans under Section 401 
of JTPA. Preceding chapters have identified a number of important influences and 
constraints on the service design decisions of Section 401 grantees, and have as well 
made implicit or explicit judgements about the effectiveness of the resulting services to 
Indian and Native American participants and communities. In this chapter we 
summarize some of these observations in a general way. 

Influences on Service Design Decisions 

As has been reiterated numerous times throughout this report, key influences on 
service designs include amount of funding, community and client needs, cultural 
factors, characteristics of the labor market, and tribal economic development, among 
other things. 

Inadequacy of Funding 

The amount of funds appropriated for the INA program is of primary concern in 
this evaluation. Because of where some of the projects are located (isolated, both 
geographically and social-culturally) , there are few employment opportunities that 
translate into appropriate training opportunities. Hence, to meet Section 401 program 
requirements, the grantees must try to be flexible and adaptable to meet the needs of 
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their clients. However, because of these situations, costs to provide needed or 
requested services are high. Moreover, access to scant employment opportunities or 
access to training may be difficult, because clients live in isolated areas. Thus, funding 
reductions, of the kind INA programs have recently experienced, easily translate into 
reduction in service delivery. 

As mentioned in an earlier section, INA 401 grantees specified goals for their 
programs that mainly reiterate the goals in DOL’s regulatory language. However, the 
sponsoring organizations, be it tribal or urban, have for the most part a more 
comprehensive philosophy— in effect, to “meet the needs of Indian/Tribal people,” in 
keeping with the cultural value “to help others.” The Section 401 program is only one 
part of their efforts to help others— a most meaningful part, but not their sole effort. 
Reduction of funds for Section 401 projects that were meager to begin with have forced 
these sponsoring agencies to shift to designs that are low cost, such as Training 
Assistance and Direct Placement. The reduction in funding thus has curtailed 
classroom training, support services, and community service employment, in quantity if 
not in quality. Work experience is still a priority in Section 401 project designs, but 
funding cuts limit the number who participate in it as well. 

Community and Client Needs 

The diverse employment training needs of prospective 401 participants are 
predicated on their need to provide for self and for their family. Further, there is a 
need, if possible, to obtain training near to them and, upon completion, obtain 
employment also near where they live. This report cites instances of 401 participants 
opting to train and work in their tribal communities, be it reservation or urban. The 
lack of employment opportunities on the reservation and client preferences to “work 
with an Indian organization” compound the grantees’ efforts to meet their clients’ 
needs. Clients are in turn faced with difficult choices that impact their successful 
termination: choosing appropriate training, obtaining employment compatible with their 
training, choosing to stay near their home community, or to relocate to obtain 
employment elsewhere. The sense of home community is strong among tribal people 
and many tribal people choose not to leave home to obtain employment or training for a 
variety of reasons. They may feel intimidated by unfamiliar surroundings, they may 
need the support of extended family (child care, transportation, etc.), they may not 
have sufficient life skills to prosper/survive in the new setting, or they may be 
unwilling to abandon the supports in their home community with which they are 
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familiar. If appropriate options are unavailable, clients thus must make difficult 
choices— to obtain employment compatible with the training regardless of its location 
and sever ties with the home community, or choose not to move and risk becoming a 
“negative” terminee. These situations taken in combination represent decisions that are 
difficult and have a lifelong impact. 

Labor Market 

The labor market in Indian country plays a major factor on project design. If 
there are employment opportunities in the projects’ service area, sites for on the job 
training and work experience and access to classroom training for a job in demand in 
the area may be available; INA services can then be designed with these opportunities 
in mind. However, if these opportunities are few or non-existent, then the grantees 
have a monumental task to meet clients’ needs. How does one train a client in a field 
in which they are interested when employment opportunities are not there? The 
grantees can provide learning experiences that are of interest to the client but may have 
difficulty placing the client in that area; put differently, regardless of the training 
received, it may be to no avail if employment cannot be obtained. Weak labor markets 
are thus a pervasive influence on all aspects of service design and delivery, from the 
grantees’ initial conceptualization of their program goals and objectives, to the types of 
services they rely on and make available to clients, and to the types of outcomes they 
can realistically expect. 

A more sensitive influence associated with weak economies that grantees confront 
is the “problem” of client motivation. Indian people face many actual and perceived 
constraints in seeking employment. In situations where job availability is negligible, 
there may be fierce competition for those jobs. In those limited job markets 
(reservations), the Tribe may be the main source of employment. Although Tribes may 
not want to admit to this practice, tribal politics, family relations or “repayment” for 
past favors may play a part in who gets employed. This may not be different than any 
employment area, but it is perceived as such in many tribal communities. Some clients’ 
efforts to obtain employment may be thwarted, regardless of their efforts to improve 
their employability through training. It is understandable then that the motivation to 
seek employment again and again could weaken one’s resolve. Alternatives to not 
being able to be employed in the home area are to leave the home area to gain 
employment. This may have been tried and the client may have not succeeded for 
reasons mentioned before. Therefore, an unemployability cycle is begun and 
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perpetuated. So, in some instances, some clients recycle themselves through the 
program time and again. 

Tribal economic development programs also play a major role in employment on 
reservations, which effects 401 project designs. Tribes who either contract or compact 
federal monies or programs can provide a variety of employment opportunities. 

Through the Work Experience program, grantees can place clients with a variety of 
tribal programs, which may be of mutual benefit. For example, the client receives 
valued work experience and the tribe receives low-cost labor to provide more services 
to the community that may or may not be available otherwise. 

As another coping strategy, Section 401 programs “depend” on partner agencies 
to provide a variety of support services that the grantee itself cannot afford. Key to the 
success of this effort is the experience and long-tenure of staff. As integral members of 
their communities, the 401 -grantee staff can serve an important role in referring clients 
to social, health, housing, food, and child care services, in an effort to meet client 
needs. These staff have developed relationships and an understanding of resources 
available, who should be contacted and what the guidelines are to obtain services. This 
“institutional” knowledge by many of the grantee staff may influence small grantees to 
invest in human resources to provide a wide variety of services vs. investing in other 
project options. 

Tribal Influences 

Tribal culture exerts itself in subtle and complex ways. For example, it is 
understood among Indian people that one doesn’t ask, “Are you Indian?” when 
interviewing a potential client... one asks “What tribe are you?” A base of 
understanding is thereby established, a starting point to make the potential client 
comfortable and ease anxiety that the client may be experiencing. Most generally the 
grantees’ offices are decorated in an Indian motif, and walls with Indian paintings and 
posters announcing Indian events are prominent. There are Indian staff persons, some 
listening to Indian songs, and other Indian clients are about. There is thus an Indian 
feel to the office, and, to a lot of Indian people, this is comfortable and accepting— a 
place where Indian people feel they belong. There is information on the availability of 
tribal programs in a variety of social services, education, housing, and health areas. As 
the Indian staff continue the intake process, the Indian environment of the office and the 
cultural identity of Indian staff make the interview easier for the program candidate. 



X-4 

263 



Cultural influence also manifests itself by the programs’ participation in Tribal- 
Indian events, such as Pow-wows, health fairs, sports events, parades, educational 
workshops, career days, and awareness events, among others. Grantee staff are 
community members and, as such, participate in community events. In many 
situations, it may be difficult for the larger Tribal-Indian community to distinguish 
grantee staff from other private members of the community. Tribal languages may be 
utilized in some project offices and this too operates as a vehicle of cultural inclusion. 

Finally, grantee staff and their clients typically know each other and each other’s 
families by virtue of being members of the same community. Thus, they have a 
person-to-person, face-to-face relationship that comes from interacting directly in daily 
life: they go to the same stores, attend the same churches, see each other at community 
events, etc. This lends a degree of personalization and intimacy not typical in other job 
training or social service systems. 

Many grantees discuss with clients cultural differences, not to orient clients in 
negative ways, but to acquaint clients with some non-Indian attitudes and expectations 
of employment. For example, some clients may be unfamiliar with the non-Indian 
perspectives of work and employment. Grantee staff thus feel a responsibility to share 
some of these perspectives, such as those relating to work attendance, work habits, 
work attitudes, inter-personal relations, and other employer perspectives. The 
emphasis of this orientation is to assist the Indian-Tribal client in operating effectively 
in the non-Indian world. 

Some Thoughts on Project Effectiveness 

The dictionary defines “effective” as “having an expected or intended 
effect... prepared for use or action... producing or able to produce a desired effect... the 
capacity or power to achieve the desired result.” Given this definition, grantees are 
achieving varying degrees of effectiveness. In the discussion below, we review some 
practices in both design and operations that are most effective overall and those which, 
although meeting some needs, are not as effective. 

Among the programs’ most effective attributes is the holistic approach to 
services, which constitutes a major plus for clients, many of whom have diverse needs 
that must be addressed in their attempt to become employed or employable. Each 
grantee in this study had a network of employers to which they could refer clients, or 
had health, social service, housing, food distribution, transportation or other services 
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that enable clients to progress in their employment and training needs. As mentioned, 
cultural influences are prevalent, not so much in actual physical manifestations, but in 
environment, attitude, and understanding the ways that clients can relate to and feel 
good about their experiences with the grantee interaction. 

Most of the grantees also had well-established systems to counsel and plan 
services for clients. As a result, clients receive individual attention, assistance, and 
monitoring as they progress through their training experience. Key, again, to the 
grantees’ operations are the project staff. Time and again, as we interviewed staff, we 
became aware of their dedication, commitment, and sensitivity to meeting clients’ 
needs. They also exhibited remarkable innovation and creativity in service planning, 
service delivery, and placing clients in employment. 

There also appears to be good to excellent relations between classroom training 
providers and the Section 401 projects. Case managers for the most part had both 
formal and informal channels of communication with their service providers and in 
some instances could contact students if they missed class, didn’t turn in a paper, or 
committed other such “minor” incidents. All this was done in an effort to assist the 
student to succeed in their training endeavors. The classroom service providers and 
case managers felt it important to “nip in the bud” student lapses by counseling with 
them. Similar efforts were made to assure that OJT, TA, CSE, and WEX clients 
maintained positive employment habits and positive attitudes during their employment 
experiences. Grantees thus endeavor to meet their client needs before the needs 
become so overwhelming that the student will dropout of the program or quit their job. 

Grantees provide many services in their attempt at a holistic approach to service 
delivery, which appears to be one of the program’s most effective elements, for the 
reasons mentioned above. Nonetheless, when looking across programs, the services 
themselves are of mixed effectiveness, with many programs demonstrating some very 
strong elements but with weaker elements also emerging. Basic skills classroom 
training, for example, is typically characterized by an open-entry/open-exit approach to 
services and is self-paced and geared to each student’s own learning deficiencies. 
However, in some cases some programs, due partly to a lack of resources, have 
adopted an unstructured approach to instruction with minimal supervision by qualified 
instructors. Only clients who exert strong self-discipline and motivation for success- 
traits that can not always be counted on among students in basic skills instruction— are 
able to make progress; others find themselves floundering. 
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Work Experience and Community Service Employment are among the most 
frequently used service activities and can be very effective in addressing a wide range 
of client needs, both short-term and long-term, including their need for immediate 
income, pre-employment and work maturity skills training, career exploration, 
occupational skills training, and a boost to self-esteem. Moreover, these service 
activities also bestow substantial benefit to the community, by making available social 
services (e.g., child care, tribal administration, health services, etc.) that the tribe or 
community could otherwise not be able to afford. They are also very versatile service 
activities from the standpoint of providing good service designs even when employment 
opportunities and alternative training options are meager. At the same time, these 
activities are very costly and in some cases WEX or CSE assignments, even those of 
substantial duration, leave the client only slightly better off at the end of the spell of 
subsidized employment than they were at the outset, particularly when grantees do not 
ensure that training objectives are clearly specified. 

Similarly, on-the-job training is used by some grantees very effectively, as when 
the OJT assignment has clear training goals and the client is. provided with intense 
supervision and even mentoring, and permanent unsubsidized employment results when 
the training period ends. But some grantees take too little pains to ensure that their 
OJT assignments follow these guidelines, as when employers do not provide good 
quality training or the training plan is too formulaic. 

In conclusion, project effectiveness is dependent on a wide variety of influences. 

If effectiveness is equated as meeting performance standards, then the projects in this 
study can be deemed to be highly effective. But not all is rosy nor are all programs 
above average. There are many positives and there are many “less than positive” 
aspects of some programs. To improve some of the less-than-positive aspects will take 
changes that may be beyond the capacity of the grantee in some cases (e.g., because of 
budget constraints) or require additional staff training. 

The constraints in which Section 401 programs operate are formidable indeed. 
How does one create more jobs in an area that may not lend itself readily to economic 
development? How does one penetrate a labor market that may be less than accepting 
of native people and tribal cultures? How does one encourage native clients who 
choose to train or be employed near home to leave their home in pursuit of other 
opportunities? The Section 401 grantees face many challenges in addressing these other 
issues. There is no doubt that the Section 401 programs provide immeasurable 
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assistance to Tribes and Indian communities; but additional input is needed in many 
cases to deem the program a total success. Perhaps DOL-Tribes-Indian organizations 
need to form alliances that can forge long-range employment plans that provide for a 
large influx of federal monies initially to stimulate economic development and, over a 
period of time, these federal monies could diminish as Indian communities realize 
economic self-sufficiency. 

Currently, the balance of federal dollars and tribal/Indian self-sufficiency is 
uneven, creating a dependency on the federal dollar. What is needed is to stimulate the 
local economy enough to generate employment and loosen dependence on the federal 
dollar. Can the 401 program better contribute to economic development? Can better 
“alliances” be formed”? Regardless, Tribal- Indian self-sufficiency needs to become a 
reality. How to do it is not clear, but it must be done if tribal communities and Indian 
people are to achieve economic parity. 
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XI CONCLUSIONS AND RECOMMENDATIONS 



Over the course of this three-year evaluation, study team members visited nearly 
two dozen grantees all around the country, including those whose JTPA allocations are 
among the very largest and very smallest, those in active urban economies and in 
economically depressed rural areas, and those run by tribal governments and non-profit 
organizations. We talked with scores of program staff and program participants and 
observed a broad array of training and employment activities, including GED classes, 
occupational skills classroom training, OJTs, work experience, community service 
employment, and training assistance. And we scrutinized the results from a very 
successful Administrator Survey, in which over 100 grantees participated, many of 
whom wrote lengthy and extensive comments on a wide range of topics. 

Based on this wide-ranging data collection effort, we have gained a good 
understanding of not only the Section 401 program’s strengths and weaknesses, but also 
what makes the program unique and, indeed, why there needs to be a separately funded 
program serving the Indian and Native American community. In this chapter, we 
summarize some of these observations and present some policy recommendations, as 
DOL moves to implement an Indian and Native American Program under the new 
Workforce Investment Act of 1998. 

Summary Observations 

The results described in the preceding chapters of this Report provide extensive 
evidence about the operation of the INA program during the last years of JTPA. 

Without intending to summarize these findings (the reader is referred to the Executive 
Summary for this purpose) , this chapter draws out some general implications of our 
findings. 

At the level of federal oversight and policy, the Department of Labor has over the 
last several years made enormous strides in forging a strong partnership with the 
grantee community. Although the degree of cooperation evidenced in the DOL-grantee 
relationship has waxed and waned over the past decades, a fundamental underlying 
sense of mistrust, even antagonism, has characterized their interactions. DINAP’s 
outgoing chief, Mr. Thomas Dowd, who is himself a Native American and a former 
grantee director, has worked hard to dispel suspicion and forge a constructive dialogue. 
Although some grantees remain wary, the last several years have as a consequence 
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witnessed an extraordinary transformation in the way DINAP is perceived. To an 
extent unforeseen only a few years ago, grantees now feel a strong sense of ownership 
towards “their” program and are working cooperatively with DINAP on all matters 
affecting program policy. Based on this foundation, the INA program is now well 
positioned to develop a sound future under the new Workforce Investment Act. 

In terms of service design and delivery, this future will doubtless reflect the 
unique role that INA programs play in the employment and training system and the 
many special strengths that were in evidence among the programs we studied. To begin 
with, INA grantees epitomize the holistic approach to services. Programs we studied 
were inordinately conscious of viewing clients as whole people rather than as instances 
of symptoms to be treated. This approach often extended to a consideration of the 
needs of other members of the participant’s family and the ways in which family 
dynamics needed to be taken into account to promote the individual’s own success. 

The notion of holistic services was expanded as well to the conceptualization of 
the grantee community as a constituency in its own right and as an important 
beneficiary of the services that were being provided to individuals. Thus, grantees 
sometimes needed to strike a balance between deciding how to meet the needs of 
individuals for employment and training assistance while ensuring that the community’s 
own larger interests were being advanced as well. The ways in which Work 
Experience and Community Service Employment are used clearly exemplify this. 

The importance of community surfaced in another way as well, particularly for 
programs operating away from a tribal setting. In such environments, participants (and 
potential participants) often feel estranged and alienated from the mainstream culture 
and look to the organization operating the Section 401 grant as providing a needed 
sense of belonging. Moreover, participants take great comfort in knowing that their 
needs for assistance can be met by program staff that share a common ethnic identity as 
Native Americans and are sensitive to tribal cultures. So important is this shared 
identity that many participants would not avail themselves of similar services that might 
be made available from a non-Native American organization. 

For their part, grantees, again particularly those in non-tribal settings, become a 
focal point for community identity. As such— and, again, reflecting their holistic 
approach to services and their focus on meeting individual as well as community 
needs— they use funds from a variety of sources to organize cultural activities and 
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events, promote native arts and crafts, run programs for senior citizens, operate Food 
Banks for the needy, publish community newsletters, and the like. The strong sense of 
community engendered by these efforts is largely irreplaceable and constitutes one of 
the strongest aspects of the uniqueness of the INA program. Although many of these 
subsidiary efforts rely on non-401 funds and a strong spirit of volunteerism, the JTPA 
allocation constitutes the bedrock for the grantees’ very existence and thus must be 
viewed as the foundation of these myriad efforts. 

Grantees operating in tribal settings, by contrast, to a much lesser degree serve as 
the community’s focal point, because so many other tribal agencies and institutions can 
play this role. Instead, the JTPA program must be viewed as constituting an integral 
cog in a wider network of services developed from other funding streams, used in 
tandem to meet the needs of individuals and the community. 

The above considerations suggest the importance of evaluating the INA program 
in context. The importance of context is suggested as well by the fact that grantees are 
operating their programs in extraordinarily diverse and often extremely difficult 
circumstances, typically with very limited funding. For example, service areas in many 
instances are marked by high rates of joblessness, physical isolation, and extreme 
economic deprivation. Fashioning an employment and training program in such 
circumstances poses special challenges and gives rise to difficult decisions regarding 
how a job training program should focus its energies in the face of a dearth of 
unsubsidized job opportunities of any kind. 

The importance of context suggests that it is very difficult to analyze specific 
aspects the Section 401 program (such as specific training activities) in isolation. In a 
real sense, in fact, the value of the program is much greater than the sum of its 
component parts. Thus, specific activities or services, when viewed in isolation, often 
seem unexceptional, but the INA programs taken as a whole play a critical role in 
promoting the vitality and well being of Indian and Native American individuals and 
communities. 

Recommendations 

Based on our extensive examination, we are convinced of the critical role that the 
INA program plays for the people and communities they serve. We applaud the efforts 
of grantees who design and deliver quality services in the face of inordinately difficult 
challenges, not the least of which is a serious shortfall of job training dollars. 
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At the same time, we have drawn on our study’s findings to formulate a number 
of recommendations, which we have formulated at the level of federal policy and 
practice and the level of the grantee service design and delivery. 

Federal Policy and Practice 

1 . DOL must take seriously, as it has begun to do over the last several years, its 
obligation to work in partnership with the grantee community. Recent initiatives 
undertaken by the Division of Indian and Native American Programs (DINAP) , in 
conjunction with the Division of Performance Management’s Office of Policy and 
Research (OPR) , have been successful in enabling grantees to feel ownership of 
their program. DOL and grantees are now negotiating in a spirit of openness and 
cooperation to make important decisions about the program’s operation and future 
direction. Given that Mr. Dowd has recently announced that he will be stepping 
aside as Chief of DINAP, DOL needs to ensure that his successor is as committed 
to dialogue and partnership as he was. The initiatives that Dowd began should not 
be allowed to falter. 

2. DOL needs to ensure that all grantees have the opportunity to participate in the 
partnership initiative. Results from our Administrator Survey suggest that some 
grantees, while applauding the partnership initiative, feel that they have not had full 
opportunity to participate. Although Work Groups and advisory bodies can 
understandably include just handfuls of members, DOL should ensure that all 
grantees have ample opportunity to express their opinions on matters affecting the 
program. Where it would not be too disruptive, Work Group or partnership bodies 
should include provisions for rotating memberships, and efforts need to be made to 
see that all grantees, including tribal and non-tribal programs and those that are 
large and small, are well represented. DOL’s recent initiative to promote Internet 
access for all grantees may provide an additional vehicle for giving all grantees the 
chance to participate in a dialogue. 

3. DOL needs to work with grantees to ensure that new regulations for the Section 401 
program are clear, concise, and grant ample flexibility to grantees to design and 
operate their programs in accordance with the needs of their communities. The 
recently enacted Workforce Investment Act (WIA) of 1998 supplants JTPA, but still 
allows a provision for a separate, national Indian and Native American Program. 
Doubtless, DINAP will need to craft new regulations for the program under WIA. 

In keeping with recommendations expressed above, new regulations should be 
developed with the full cooperation of the grantee partners. In recognition of the 
great diversity of contexts within which the IN A program operates, we recommend 
that these regulations permit ample flexibility to grantees for designing programs 
that are responsive to local needs. Overly restrictive provisions on service designs 
that are not statutorily mandated should be avoided. Any new regulations, as well 
as all other DOL issuances (e.g., DINAP Bulletins), should also be written with an 
eye to simplicity and clarity. 
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4. The new regulations need to permit WIA funds to be used to promote diverse 
individual and community needs, including economic development if possible. Lack 
of adequate job opportunities and weak economies are among the biggest obstacles 
Section 401 grantees face in accomplishing their program objectives. Given these 
circumstances, concerted efforts should be made to spur economic development in 
Native American communities. The JTPA Amendments of 1992 placed severe 
restrictions on the use of JTPA funds for economic development activities, 
effectively eliminating a promising Community Benefits Projects (CBP) initiative 
that had previously been permitted in the INA program. 1 To the extent that it is 
allowable under WIA, we encourage DOL to consider reinstating provisions for 
Community Benefits Projects, or similar initiatives to allow WIA funds to be used 
to promote, or in conjunction with, economic development efforts. 

5. DOL should forge ahead with plans to revamp the performance standards system for 
the INA program, to reflect the wide diversity of grantees' circumstances and 
accomplishments. At the same time, it must be careful to ensure adequate 
accountability at the national level. Grantees are clearly ambivalent about the 
current performance measurement system. Although the current measures are 
generally perceived as being reasonable and fair, many grantees also feel these 
measures do not fully reflect their programs’ chief accomplishments. Thus, DOL’s 
recent efforts to work in partnership with the grantee community to develop a 
revised performance-standards system seem wholly appropriate. The recent 
enactment of the Workforce Investment Act should not delay these efforts. 

The twin themes of this new legislation are, on the one hand, to devolve 
substantial authority for decision-making to local programs, and, on the other, to 
ensure that local programs are held strictly accountable for their performance. In 
keeping with these themes, DOL should ensure that new performance measures (as 
well as program regulations in general) impart substantial flexibility, while at the 
same time ensuring that adequate accountability mechanisms are in place. This will 
require that any new performance-standards system provide meaningful and 
substantial measures of accountability. Additionally, the new measures need to be 
supported by an adequate reporting vehicle that includes clear definitions of key 
terms and the valid and reliable measurement and reporting of key concepts. 

6. DOL needs to ensure that additional attention is paid to the needs of grantees for 
technical assistance and training, especially those who are new Directors, and that 
grantees have adequate opportunity to engage in dialogue with their peers and 
DINAP’s Federal Representatives. Any new flexibility imparted under WIA implies 
as well that grantees be provided with sufficient opportunities to learn how to take 
advantage of that flexibility by designing effective and innovative services. Along 
these lines, although the technical assistance and training (TAT) that has been 



1 The provisions of CBP allowed grantees to use a limited amount of their JTPA funds to engage 
in workforce development and training in the context of promoting local economic development. 
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provided heretofore has been adequate overall, certain segments of the grantee 
population, especially new Directors, have not received sufficient guidance. 

In general, capacity building needs to also be promoted, and grantee staff 
should have the opportunity to build their skills in all areas. Assessment is a 
particular area in which programs might benefit from additional capacity in the 
years ahead. Participants will increasingly need to improve their skills to compete in 
the labor market; schools’ and employers’ expectations and requirements have risen 
over the past decade and will continue to place more importance on excellent 
academic and workplace skills into the next century. In this context, a careful 
assessment should be the foundation upon which participants’ services are based. 
Without a thorough knowledge of participants’ interests and abilities, service plans 
can be made based only on a general feeling for the clients’ capabilities and long- 
term needs, and thus risk missing the mark. Given this, staff will need to have 
training on how best to access assessment results and how to interpret them. 

Grantees could also benefit from efforts to learn about what their colleagues are 
doing, including through peer-to-peer exchanges. DOL should also continue its 
recent efforts to ensure that grantees have adequate opportunity to meet individually 
with DINAP’s Federal Representatives. 

Grantee Service Design and Delivery 

7. To ensure more equitable access to services, grantees should avoid an over-reliance 
on word-of-mouth referrals. For the same reason, they need to make provisions for 
reaching potential applicants throughout their service areas, through outstationing 
staff and forging partnerships with other social service agencies. Outreach and 
recruitment efforts currently being undertaken are clearly adequate for ensuring a 
constant flow of participants. However, some grantees rely almost exclusively on 
indirect recruitment methods, especially word-of-mouth referrals, and this has 
sometimes made it difficult for potential applicants who are less well connected in 
the community to be made aware of services. Thus, some participants with whom 
we spoke, who eventually heard about the program through a friend or relative, told 
us that they wished they had learned of the program’s existence years earlier. 

An additional difficulty that grantees experience in ensuring an equitable access 
to services is in reaching potential participants throughout the entirety of the 
grantees’ service areas. Achieving this objective is an especially difficult challenge 
for those grantees serving physically vast territories but who find it financially 
infeasible to establish separate field offices, due to their very limited funding. In 
such cases, grantees can follow the lead of many of their colleagues who outstation 
staff, use roving recruiters, and develop effective joint referral linkages with other 
social service and tribal programs. As the new WIA legislation takes hold, linkages 
with the nation’s emerging One-Stop systems can be an especially promising 
practice that is rife with opportunity, as a few grantees have already demonstrated. 
Thus far such linkages appear to be underutilized. 

8. While grantees need to attend to participants ’ immediate needs for income and 
employment, they need to address clients ’ longer-term needs as well. We found that 
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clients often had strong opinions about what services they wanted and needed in the 
short-run, typically including obtaining immediate employment (subsidized or 
unsubsidized), driven by their need for immediate income. Consequently, Section 
401 grantees often emphasized addressing participants’ shorter-term needs, 
sometimes to the neglect of their longer term needs, and resulting in quick-fix 
solutions and a “revolving door” approach to services. We recognize that grantees 
often find themselves severely constrained by limited budgets and other factors. We 
also appreciate that they must be responsive to the expressed preferences of their 
clients. To the fullest extent possible, however, grantees should promote long-term 
solutions and structure service strategies to advance participants’ longer-term 
interests while attempting to address their needs in the short run. Suggestions for 
how to do so are embedded in some of the recommendations that follow. 

9. Grantees who are not located in reasonable proximity to service providers for 
Classroom Training can utilize distance learning or alternative delivery vehicles. 
Extreme physical isolation is a major impediment to making the full-range of 
classroom training services available to participants, at least among some grantees. 
Alternative or innovative service delivery vehicles can be pursued in these 
circumstances, including distance learning or other on-line classroom services. The 
greater access to technology among both grantees and participants will make this 
more and more feasible. 

10. Grantees need to ensure that On-the-job Training is accompanied by a clear 
training plan, specifying the specific skills the participant is expected to learn, and 
in fact provides training opportunities commensurate with the employers ’ wage 
reimbursements. On-the-job training (OJT) has great potential as a service activity 
in imparting meaningful skill gains and leading to permanent employment, and to 
this degree can clearly be said to address participants’ longer-term, as well as short- 
term, needs. However, to realize this potential to the fullest, grantees must ensure 
that the participant’s training plan is clearly specified, that the work assignment 
imparts meaningful skills that are transferable across employers, and that the 
employer provides adequate supervision and mentoring. Moreover, grantees should 
have the expectation that employers will continue to hire the trainee once the 
training period has ended, and limit their involvement with employers who fail to 
do so. Many of the OJTs we studied exemplified these characteristics, but many 
others did not. Grantees should ensure that all OJTs attempt to promote high- 
quality design principles. 

1 1 . Work Experience assignments of substantial duration should also be structured to 
provide real and meaningful training. Work Experience (WEX) serves very diverse 
objectives and is used by grantees in very different ways. In some cases, WEX is 
deliberately and appropriately structured as a short-term or stopgap work 
assignment, as when participants are given short-term WEX positions while 
undergoing job search or as a way of providing exposure to the world of work and 
boosting self-esteem. Very often, however, WEX assignments are of substantial 
duration (e.g., up to the 6-month limit). In these cases, grantees should ensure that 
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training objectives are clearly specified and the participants’ are learning valuable 
occupational skills. By doing so, grantees can again promote participants’ long- 
term needs for quality skill development, as well as their short-term need for 
immediate income. 

12. For participants interested simply in direct placement assistance, grantees need to 
ensure that they provide not only job referrals, but also that they build participants ' 
job search skills. Section 401 grantees very appropriately often provide direct 
placement assistance. In doing so, they should avoid simply giving job referrals, 
especially when participants seem to lack good job search skills. In such cases, 
grantees that provide training in job search skills boost participants’ self sufficiency 
and empower them to seek and find their own jobs in the future, minimizing their 
subsequent need for the program’s assistance. 
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Appendix A 

The Administrator Survey 



SECTION 401 JTPA 
PROGRAM ADMINISTRATOR SURVEY 



Dear Grantee Partner: 

The Native American Employment and Tr aining Council (the Advisory Council) and the 
Department of Labor’s Division of Indian and Native American Programs (DINAP) agreed to 
have an evaluation conducted of the Job T raining Partnership Act, Section 401 Indian and Native 
American Program, to better understand the program’s effectiveness and participant outcomes. 

This comprehensive study is being conducted by Social Policy Research Associates and American 
Indian Research and Development. The study has two components. The first, was completing a 
series of randomly selected program site evaluations; all have been completed. The second and 
final component of the study is my reason for writing this letter. I am asking for your 
cooperation in completing the enclosed voluntary survey. 

The survey is intended to assess grantee satisfaction with the services and guidance received from 
the Department of Labor, to learn the greatest obstacles grantees face in obtaining successful 
outcomes for participants, and to learn about factors determining grantee service strategies. The 
survey is designed to be completed by the Section 401 JTPA program a dminis trator and includes 
questions divided into three sections: I. Program Goals and Organization Structure; II. Services 
Provided by the Department of Labor; and HI. Services Provided to Participants. Please 
complete each section and return the questionnaire in the enclosed postage-paid envelope wi thin 
the next two weeks. 

You can be assured that your answers will be kept strictly confidential and not shared with any 
person associated with the DINAP program office. The Department will receive a final report 
that does not identify source documentation by any survey contributor. 

I want to emphasize that your participation is critical to the process of continuing to improve our 
partnership based on trust, respect and co mmi tment to purpose. The survey results will 
ultimately help the Advisory Council and DINAP with improving our services to you and your 
participants, the ultimate customers. 

Thank you in advance for helping to improve our partnership by completing and returnin g the 
survey. Any questions regarding the survey should be referred directly to Ron D’Amico of 
Social Policy Research Associates, at 415-617-8628. Questions about the study in general can be 
directed to Dr. D’Amico or to Stu Tonemah of American Indian Research and Development, at 
405-364-0656. 



Sincerely, 




THOMAS M. DOWD 
Chief, Division of Indian 
and Native American Programs 



BEST COPY AVAILABLE 



JTPA Section 401 
Program Administrator’s Survey 



About the survey 

This survey about the JTPA Section 401 Indian and Native American Program is being 
conducted by Social Policy Research Associates and American Indian Research and 
Development for the U.S. Department of Labor. The survey was developed with input from 
members of the Native American Employment and Training Council and other members of the 
grantee community . Its purposes are to assess grantee satisfaction with the guidance and 
services received from the Department of Labor and to learn about grantees’ service strategies 
and the obstacles they face in meeting the needs of their participants. Your opinions are very 
important to us in providing the Department of Labor with an accurate picture of grantees’ 
concerns. Therefore, we would appreciate very much if you would take the tim<» to answer the 
survey and return it to us in the postage-paid envelope within the next 2 weeks. Please note 
that your answers will be kept strictly confidential and will be made available to the Department 
of Labor only in the form of summary tabulations, where your answers will be combined with 
those of other grantees. 

Thank you very much for your help. 



FOR FURTHER INFORMATION 

If you have any questions about the survey, please feel free to call: 

Ronald D’Amico, Social Policy Research Associates 
(650) 617-8628, or 

Stuart Tonemah, American Indian Research and Development 
(405) 364-0656, or 

Thomas Dowd, Chief, Division of Indian and Native American Programs, 
U.S. Department of Labor, (202) 219-8502 
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OMB Approval Number: 1205-0380 
Expiration Date: 2/28/98 

Part I: Program Goals and Organization Structure 

The questions in this section ask some information about your organizational structure, sources of 
funds, and the goals you have established for your JTPA Section 401 Indian and Native American 
Program. Some of these questions ask specifically about funding your organization accessed in 
program year (PY) 1996, which ran from July 1, 1996 to June 30, 1997. 

Staffing 

1 . How many people work for your JTPA Section 401 Program, including those in all field offices, 
whose salaries are wholly or partly paid for by JTPA?: 

(PLEASE FILL IN NUMBERS) 

full-time staff whose salaries are wholly paid for by JTPA (excluding work 

experience/community service participants) 

part-time staff, or full-time staff whose salaries are only partly paid for by JTPA 

(excluding work experience/community service participants) 

work experience or community service participants working in your offices full- or 

part-time 

Other Sources of Funds 

2. In addition to your Section 401 funds, does your organization receive grants from other sources to 
provide education, training, or social services for people in your community? 

(PLEASE CIRCLE ONE NUMBER) 



Yes 2 (continue with question 3) 

No 1 (skip to question 6) 



IF YES TO QUESTION 2: 

3. From how many separate sources did your organization receive funds in PY 9 6, including your 
Section 401 JTPA grant? 

(PLEASE FILL IN NUMBER) 

separate sources of funds 



4. What is the total dollar amount of these funding sources in PY 96, including your JTPA grant? 
(PLEASE FILL IN NUMBER) 

$ dollars from all funding sources 



Persons are not required to respond to this collection of information unless it displays a currently valid OMB control number. 
We estimate that it will take an average of 30 minutes per response, including the time for reviewing instructions, searching 
existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of 
information. Send comments regarding this burden estimate or any other aspect of this collection of information, including 
suggestions for reducing this burden, to the U.S. Department of Labor, Office of Policy and Research, Room No. N-5637, 
Washington, D.C. 20210 (Paperwork Reduction Project 1205-0 380, expires 2/28/98). 
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5. What are the non-Section 401 sources of money used for? 

(PLEASE CIRCLE ALL THAT APPLY) 

1 Health services 

2 Legal services 

3 Food distributions 

4 Family services (for example, family counseling, adoption services) 

5 Educational services (for example, tutoring) 

6 Job training 

7 A Seniors program 

8 Community economic development 

9 Other 



Program Goals 

6. INA grantees manage and operate their programs in circumstances that are generally unique to 
the population served and within a Native American and Indian cultural context. Given the context 
within which your program operates, what would you say are the most important goals for your 
JTPA Section 401 program? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. Providing participants with training 

b. Helping participants find jobs 

c. Helping participants achieve self-sufficiency 

d. Assisting participants with their supportive 
needs (for child care, counseling, 
transportation assistance, and so on) 

e. Providing participants with temporary income 
while they are in training or subsidized 
employment 

f. Providing a place in the community where 
people know they can come for help 

9- Other (specify) 



Extremely 

Important 

4 

4 

4 

4 

4 

4 

4 



Quite 

Important 

3 

3 

3 

3 

3 

3 

3 



Somewhat 

Important 

2 

2 

2 

2 

2 

2 

2 



Not At All 
Important 

1 

1 

1 

1 

1 

1 

1 



2 
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Part II: Services Provided by the Department of Labor 

The questions in this section ask you to assess the responsiveness of the services provided by the 
U.S. Department of Labor's (DOL) Employment and Training Administration, including the Division of 
Indian and Native American Programs (DINAP). We are especially interested in knowing what DOL 
and DINAP could be doing differently to help you better meet the needs of your participants and 
community. Of course, your answers are completely confidential; no office within DOL will know how 
you respond. 



Policy Guidance 

7. As part of its grant management responsibilities, DINAP is respon sible for developing program 
direction for Section 401 grants . This includes setting priorities that result in making the program 
easier to understand and implement (e.g., streamlining regulations, improving reporting 
requirements, approving general waivers, etc.). How would you rate the following statements 
about the policy guidance provided by DINAP? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. DINAP clearly communicates the program’s 
policies 

b. Our program has enough flexibility under the 
JTPA grant to address the needs of our 
participants 

c. Program regulations are too complex 

d. The recent partnership efforts undertaken by 
DINAP to change the program regulations 
are greatly needed 

e. DINAP’s current partnership initiative makes 
it easier for my program to have a say in 
decisions regarding program policies 



Agree 


Agree 


Disagree 


Disagree 


No 


Strongly 


Somewhat 


Somewhat 


Strongly 


Opinion 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 



Technical Assistance 

8. DINAP, in partnership with the National Indian and Native American Employment and Training 
Conference and grantee regional organizations, co-sponsors training and technical assistance 
designed to assist grantees to improve their program management and provide better service to 
participants. DINAP also periodically issues Bulletins and maintains a Web site, and its staff are 
available to provide support over the phone. How would you rate the following statements about 
this technical assistance? 



(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. The technical assistance at the national 
conferences meets the needs of grantees 

b. The one-on-one (Fed. Rep./Grantee) 
sessions held at conferences help to meet 
grantees’ specific needs 



Agree 


Agree 


Disagree 


Disagree 


No 


Strongly 


Somewhat 


Somewhat 


Strongly 


Opinion 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 
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8. Technical Assistance (continued) 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



c. The regional conferences provide a good 
opportunity to get assistance from fellow 
grantees 

d. I support the use of peer-to-peer (grantee-to- 
grantee) technical assistance 

e. The DINAP Bulletins provide important 
information about program requirements 

f. I access the Internet to obtain information 
from DINAP that benefits my program 



Agree 

Strongly 


Agree 

Somewhat 


Disagree 

Somewhat 


Disagree 

Strongly 


No 

Opinion 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 



g. DINAP’s partnership efforts have increased 
my willingness to ask questions from the Fed 
Reps when needed 

h. DINAP staff provides accurate information 
when our program asks specific questions 



i. DINAP answers our questions promptly 



4 3 2 



9 



9. Please describe how technical assistance can be improved. 
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Reporting 

10. As part of the exchange of information between DINAP and the grantees, grantees are required to 
submit periodic reports, including planning documents, semi-annual reports, and the Annual 
Status Report. As part of the recent partnership initiative, many changes were made to this 
reporting process to improve the accuracy and timeliness of reporting. These changes included 
switching from quarterly to semi-annual reporting and eliminating unnecessary line items on the 
Annual Status Report. How would you rate the following statements about the reporting 
process? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. The recent revisions to the reporting forms 
make them easier to use 

b. The reporting forms are still too complicated 

c. The Annual Status Report covers all the 
information that needs to be reported 

d. Submitting reports electronically is very 
convenient 



Agree 


Agree 


Disagree 


Disagree 


No 


Strongly 


Somewhat 


Somewhat 


Strongly 


Opinion 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 



1 1 . Please describe how the reporting process can be improved. 



Consultation 

12. The partnership efforts undertaken by DINAP, grantees, and the Native American Employment 
and Training Advisory Council have attempted to improve communication and dialogue between all 
partners. How would you rate the following statements about the consultation process? 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



The partnership effort has made working 
with DINAP a more positive experience 


Agree 

Strongly 

4 


Agree 

Somewhat 

3 


Disagree 

Somewhat 

2 


Disagree 

Strongly 

1 


No 

Opinion 

9 


The partnership effort has established trust 
and respect between DINAP and grantees 


4 


3 


2 


1 


9 


DINAP does a good job of getting opinions 
from grantees before making decisions 


4 


3 


2 


1 


9 


The Internet is a good way of giving and 
receiving information 


4 

5 


3 


2 


1 


9 



0 
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13. Please describe how the consultation process can be improved. 



MANAGEMENT INFORMATION SYSTEMS 

14. How useful to you would it be if DOL were to develop an automated Management Information 
System (MIS) computer software for free distribution to grantees? This software would be for you 
to use for your own purposes on your computers for: entering information about participants at 
intake, tracking changes in status, recording case notes, recording outcomes, and producing 
reports. 

(PLEASE CIRCLE ONE NUMBER) 



Extremely useful 4 

Quite useful 3 

Somewhat useful 2 

Not at all useful 1 

Not sure 9 



Information technology project 

Please answer the following questions about the Information Technology Project. 

15. We are an “on-line partner.” 

(PLEASE CIRCLE ONE NUMBER) 



Yes 2 (continue with question 16) 

No 1 (skip to question 18) 

Not sure 9 (skip to question 18) 



6 
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IF YES TO QUESTION 15: 

16. We check our e-mail at least once a week. 
(PLEASE CIRCLE ONE NUMBER) 



Yes 2 

No 1 

Not sure 9 



17. Please answer the following additional questions about the Information Technology Project. 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. I enjoy receiving the DINAP “Hot News 
Flash” 

b. The DINAP Home Page and Netforum are 
very useful to grantees 



Agree 


Agree 


Disagree 


Disagree 


No 


Strongly 


Somewhat 


Somewhat 


Strongly 


Opinion 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 



Performance Standards 

18. Currently, performance standards in the Indian and Native American program are established for 
three outcomes— the entered employment rate, the employability enhancement rate, and the 
positive termination rate. Grantees are expected to meet minimum standards established by DOL 
on at least 2 of these 3 outcomes. As part of the partnership initiative, DINAP is currently meeting 
with a Work Group to assist in evaluating these measures and perhaps modifying them. To 
inform this process, it would be helpful if you would answer the following questions. How much do 
you agree or disagree with the following statements about performance standards?. 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. I understand the current performance 
standard system 

b. Overall, our program is satisfied with the 
current performance standards system 

c. Performance standards provide targets that 
help us improve our program performance 

d. The model-adjustments DOL uses to 
establish minimum standards for each 
grantee are too complicated 

e. The process DOL uses to adjust 
performance standards is fair 

f- The performance measures used by DOL do 
not adequately reflect the goals or 
accomplishments of our program 



Agree 


Agree 


Disagree 


Disagree 


No 


Strongly 


Somewhat 


Somewhat 


Strongly 


Opinion 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 


4 


3 


2 


1 


9 



7 
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18. Performance standards (continued) 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



9- Performance standards established by DOL 
focus too much on job placements 

h. The use of employability enhancements as 
an outcome has helped us focus on the 
need of our participants for longer-term 
training 

i- The definitions of employability 

enhancements are too difficult to understand 

j. The current outcomes give us enough 
flexibility in running our program 



Agree Agree Disagree Disagree 

Strongly Somewhat Somewhat Strongly 

4 3 2 1 

4 3 2 1 



4 3 2 1 

4 3 2 1 



No 

Opinion 

9 

9 



9 

9 



19. How would you improve the measurement of performance used by DOL? 



Overall helpfulness 

20. Overall, how helpful is the assistance you received from DINAP in accomplishing your program 
objectives? 

(PLEASE CIRCLE ONE NUMBER) 



Extremely helpful 4 

Quite helpful 3 

Somewhat helpful 2 

Not at all helpful 1 



21 . What additional services or assistance would you like from DOL but have not received? 
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Other DOL Functions 

22. The questions in this section ask you about other offices within the Employment and Training 
Administration of DOL that have some responsibility for administering JTPA grants. How satisfied 
are you with the services you receive from these other offices? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



Very 

Satisfied 

a- The Office of Grants and Contract ^ 

Management, Div. of Acquisition & 

Assistance 

b. The Division of Resolution ^ 

and Appeals 

c. The Closeout Unit 4 

d. The Office of Performance 4 

Management and Evaluation 



Somewhat 


Somewhat 


Very 


No 


Satisfied 


Dissatisfied 


Dissatisfied 


Opinion 


3 


2 


1 


9 



3 


2 


1 


9 


3 


2 


1 


9 


3 


2 


1 


9 



Part III: Services Provided to Participants 

The questions in this final section ask you to describe the services that you generally provide to 
participants and to note the factors that make it difficult for them to succeed. 

Assessment and Service planning 

23. In what areas do you assess your participants before they are assigned to a specific training or 
other service activity? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

Assessed Assessed Assessed 







for All 
Clients 


for Most 
Clients 


for Some 
Clients 


Not 

Assessed 


a. 


Reading skills 


4 


3 


2 


1 


b. 


Math skills 


4 


3 


2 


1 


c. 


Occupational interests 


4 


3 


2 


1 


d. 


Occupational skills or aptitudes 


4 


3 


2 


1 


e. 


Previous work experience 


4 


3 


2 


1 


f. 


Barriers to participation in training or 
employment 


4 


3 


2 


1 


g- 


Life Skills 


4 


3 


2 


1 


h. 


Other 


4 


3 


2 


1 




(please specify) 









O 
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24. What methods of assessment are used for each of these areas? 



a. Reading skills 

b. Math skills 

c. Occupational interests 

d. Occupational skills or aptitudes 



Informal Both Formal 
Formal Methods/ & Informal Not 

Tests Interviews Methods Assessed 



4 3 2 1 

4 3 2 1 



4 3 2 1 



4 3 2 1 



25. How influential are the following factors in determining which types of ser vices participants will 
receive? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

Extremely Quite Somewhat Not At All 
Influential Influential Influential Influential 



a. Results of basic skills assessment 4 

b. Results of assessment of interests or 4 

aptitudes 

c. Availability of training services in the area to 4 

meet participant's needs 

d. Participant’s own preferences 4 

e. Participant’s ability/inability to support self . 

through training 

f- Lack of program funds to offer the types of 4 

training the participant might really want or 
need 

9- Lack of child care that makes it difficult for . 

participant to undertake training 

h. Lack of transportation that makes it difficult 4 

for participant to undertake training 

i. Other 4 



3 

3 

3 

3 

3 

3 



3 

3 

3 



2 

2 

2 

2 

2 

2 



2 

2 

2 



1 

1 

1 

1 

1 

1 



1 

1 

1 
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(please specify) 



We now have a number of questions about the services that participants received who terminated 
during PY 96, the program year that ran from 7/1/96 to 6/30/97 . Note that these questions focus on 
PY 96 terminees only, not all program participants. 

26. According to your Annual Status Report, the number of persons who terminated from your 
program in PY 96 was 

persons terminated in PY 96 

27. Among persons who terminated in PY 96, how many were served with training assistance 
and/or supportive services BUT NO OTHER training activity or subsidized employment? 
(PLEASE FILL IN NUMBER) 

persons terminated with training assistance or supportive services ONLY , or 

DK (Circle if you don’t know) 



28. Among persons who terminated in PY 96, how many received the following services? 
(PLEASE FILL IN NUMBER OF TERMINEES ON EACH LINE) 

Basic skills classroom training 

Occupational skills classroom training 

On-the-job training 

Work experience 

Community service employment 

DK (Circle if you don’t know) 



29. Among persons who terminated in PY 96, how many received more than one of the services 
identified in question 28? 

(PLEASE FILL IN NUMBER) 

persons terminated in PY 96 after receiving more than 1 type of training or 

subsidized employment, or 

DK (Circle if you don’t know ) 




Please Continue on the next Page 
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30. What proportion of your PY 96 JTPA terminees received the following services? 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 





Almost 

All 


Many 


Some 


Almost 

None 


Not 

Sure 


Transportation assistance 


4 


3 


2 


1 


9 


Health care 


4 


3 


2 


1 


9 


Family care (including child care) 


4 


3 


2 


1 


9 


Personal or family counseling 


4 


3 


2 


1 


9 


Housing or rental assistance 


4 


3 


2 


1 


9 


Relocation assistance 


4 


3 


2 


1 


9 


Tools, equipment, and clothing 


4 


3 


2 


1 


9 


Meals and other nutritional 
assistance 


4 


3 


2 


1 


9 


Stipends or help in obtaining financial 
assistance 


4 


3 


2 


1 


9 



31 . How important are each of the following factors in making it difficult for you to meet the needs of 
your JTPA Section 401 participants? 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



a. Restrictions and limitations imposed by DOL on 
how our JTPA Section 401 funds can be spent 

b. JTPA Section 401 performance standards 
requirements 

c. Lack of adequate training providers in the area 
who can serve my clients 

d. Lack of adequate job opportunities in the area 

e. Lack of funds to provide the services our 
participants really need 



Very 

Important 


Quite 

Important 


Somewhat 

Important 


Not Very 
Important 


4 


3 


2 


1 


4 


3 


2 


1 


4 


3 


2 


1 


4 


3 


2 


1 


4 


3 


2 


1 
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How important are each of the following characteristics of your participants in explaining why 
some of them don't achieve their goals while in your program? Participants: 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 




Very 

Important 


Quite 

Important 


Somewhat 

Important 


Not Very 
Important 


Lack motivation 


4 


3 


2 


1 


Don’t understand what it takes to get and keep a 
good job 


4 


3 


2 


1 


Lack the reading and writing skills they need to 
succeed 


4 


3 


2 


1 


Have family responsibilities or problems that 
make it difficult for them to complete training or 
maintain employment 


4 


3 


2 


1 


Lack transportation to training or employment 


4 


3 


2 


1 


Have problems with substance abuse 


4 


3 


2 


1 


Are reluctant to pursue distant job opportunities 
because of their community ties 


4 


3 


2 


1 


Have difficulty regularly attending training or 
employment activities due to their desire to 
attend cultural ceremonies 


4 


3 


2 


1 


Are gang members, which interferes with their 
motivation to succeed 


4 


3 


2 


1 


Lack work experience or good work skills 


4 


3 


2 


1 


Lack the necessary life skills 


4 


3 


2 


1 


Other 


4 


3 


2 


1 


(please specify) 











Outcomes 

33. Of participants placed in jobs at termination in PY 96, what percentage were placed in: 
(PLEASE FILL IN PERCENTAGES) 



Tribal Government or Agencies % 

Tribal Enterprises % 

Non-Tribal or Off-Reservation 

Agencies or Enterprises % 
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Miscellaneous 

34. The space below is provided for you to write your comments on any issues you want to bring to 
our attention. Other than what has been covered above, what else do you think we should know 
about your program? What other suggestions or recommendations would you like to make to 
improve the effectiveness of the Section 401 program? 



35. OPTIONAL: Please provide your name and title (your identity and all your responses will be kept 
strictly confidential!) 

Name: 

Title: 



Thank you very much for your participation. 

Your survey responses will be analyzed as part of the study of the Section 401 program being 
conducted by Social Policy Research Associates and American Indian Research and Development. 
Your responses will be held strictly confidential. Please return this completed questionnaire in the 
enclosed postage-paid envelope to: 

Social Policy Research Associates 
200 Middle field Road, Suite 100 
Menlo Park, CA 94025 
Voice Telephone: 415-617-8625 
Facsimile Telephone 415-617-8630 
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Appendix B 

Client Characteristics and Outcomes, as 
Reported from the Annual Status Report 
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INA Program 

National Summary and Time Trends 





PY 93 


PY 94 


PY 95 


PY 96 


PY 97 


Termination Summary: 


Participants 


24,924 


24,425 


22,461 


20,245 


19,752 


Terminees 


19,434 


19,383 


17,990 


16,176 


15,388 


Job placements 


10,072 


9,734 


9,047 


8,364 


8,196 


Employability enhancements 


9,943 


9,829 


8,926 


8,531 


7,829 


Enhancements with a job 


3,808 


3,615 


3,068 


3,233 


3,018 


Enhancements ONLY 


6,135 


6,214 


5,858 


5,298 


4,811 


Positive terminations 


16,207 


15,948 


14,905 


13,662 


13,007 


Outcome Summary: 


Entered employment rate (%) 


51.8 


50.2 


50.3 


51.7 


53.3 


Enhancement rate (%) 


51.2 


50.7 


49.6 


52.7 


50.9 


Positive termination rate (%) 


83.4 


82.3 


82.9 


84.5 


84.5 


Average hourly wage of those 


$6.45 


$6.53 


$7.01 


$7.08 


$ 7.42 


placed in jobs 

Terminee Characteristics (%): 


Dropouts 


20.9 


20.5 


20.8 


20.0 


20.9 


Long-term unemployed 


35.7 


37.0 


38.9 


38.7 


37.4 


Public assistance recipients 


33.1 


32.7 


34.2 


34.0 


32.9 


Multiple barriers 


31.1 


33.7 


39.1 


42.9 


39.7 


Cost Summary: 2 


Total Costs (in millions) 


$59.1 


$60.1 


$55.4 


$50.5 


$49.6 


per terminee 


$2,408 


$2,452 


$ 2,447 


$2,508 


$2,592 


per employment 


$4,646 


$4,882 


$4,865 


$4,850 


$4,867 


per positive termination 


$2,887 


$2,980 


$2,953 


$2,969 


$3,067 


Total Number of Grantees with Data 


175 


171 


174 


164 


158 



NOTE: Some grantees are excluded from the calculations each year due to missing data. Additionally, grantees 
participating in Public Law 102-477 are exempted from DOL’s reporting requirements and, therefore, are not 
included in these tables. Because the number of grantees participating in this program has risen each year, 
aggregate counts compared over time exaggerate the appearance of a downward trend. 

Average wages were weighted by grantee size and exclude grantees reporting values of zero. 

2 Total Expenditures represent aggregate expenditures (in actual dollars); per terminee, per employment, and per 
positive termination represent aggregate expenditures, less expenditures for administration and community 
benefit projects, divided by the aggregate counts of terminees, job placements, and positive terminations, 
respectively. 
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LIST OF ABBREVIATIONS 



ABE 


Adult basic education 


BIA 


Bureau of Indian Affairs 


BSCRT 


Basic skills classroom training 


CBP 


Community benefit projects 


CETA 


Comprehensive Employment and Training Act 


CRT 


Classroom training 


CSE 


Community services employment 


DINAP 


Division of Indian and Native American Programs, DOL 


DOL 


U.S. Department of Labor 


EEN 


Employability enhancement rate 


EER 


Entered employment rate 


GED 


Graduate equivalency diploma 


INA 


Indian and Native American 


JOBS 


Job Opportunities and Basic Skills Program 


JTPA 


Job Training Partnership Act 


OJT 


On-the-job training 


OSCRT 


Occupational skills classroom training 


PTR 


Positive termination rate 


PY 


Program year 


TA 


Training assistance 


TAT (TA) 


Technical assistance and training 


TERO 


Tribal Employment Rights Office 


WEX 


Work experience 


WIA 


Workforce Investment Act 



O 

ERIC 
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